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PREFACE 


Linear Programming deals with the problem of minimizing or maximizing a 
linear function in the presence of linear equality and/or inequality constraints. 
Since the development of the simplex method by George B. Dantzig in 1947, 
linear programming has been extensively used in the military, industrial, 
governmental, and urban planning fields, among others. The popularity of linear 
programming can be attributed to many factors including its ability to model 
large and complex problems, and the ability of the users to solve such problems 
in a reasonable amount of time by the use of effective algorithms and modern 
computers. 

During and after World War II, it became evident that planning and 
coordination among various projects and the efficient utilization of scarce 
resources were essential. Intensive work by the United States Air Force team 
SCOOP (Scientific Computation of Optimum Programs) began in June 1947. As 
a result, the simplex method was developed by George B. Dantzig by the end of 
the summer of 1947. Interest in linear programming spread quickly among 
economists, mathematicians, statisticians, and government institutions. In the 
summer of 1949, a conference on linear programming was held under the 
sponsorship of the Cowles Commission for Research in Economics. The papers 
presented at that conference were later collected in 1951 by T. C. Koopmans 
into the book Activity Analysis of Production and Allocation. 

Since the development of the simplex method, many researchers and 
practitioners have contributed to the growth of linear programming by 
developing its mathematical theory, devising efficient computational methods 
and codes, exploring new algorithms and new applications, and by their use of 
linear programming as an aiding tool for solving more complex problems, for 
instance, discrete programs, nonlinear programs, combinatorial problems, 
stochastic programming problems, and problems of optimal control. 

This book addresses linear programming and network flows. Both the 
general theory and characteristics of these optimization problems, as well as 
effective solution algorithms, are presented. The simplex algorithm provides 
considerable insight into the theory of linear programming and yields an effi- 
cient algorithm in practice. Hence, we study this method in detail in this text. 
Whenever possible, the simplex algorithm is specialized to take advantage of the 
problem structure, such as in network flow problems. We also present 
Khachian’s ellipsoid algorithm and Karmarkar’s projective interior point 
algorithm, both of which are polynomial-time procedures for solving linear 
programming problems. The latter algorithm has inspired a class of interior 
point methods that compare favorably with the simplex method, particularly for 
general large-scale, sparse problems, and is therefore described in greater detail. 
Computationally effective interior point algorithms in this class, including affine 
scaling methods, primal—dual path-following procedures, and_predictor— 
corrector techniques, are also discussed. Throughout, we first present the 
fundamental concepts and the algorithmic techniques, then illustrate these by 
numerical examples, and finally provide further insights along with detailed 
mathematical analyses and justification. Rigorous proofs of the results are given 
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without the theorem—proof format. Although some readers might find this 
unconventional, we believe that the format and mathematical level adopted in 
this book will provide an insightful and engaging discussion for readers who 
may either wish to learn the techniques and know how to use them, as well as 
for those who wish to study the theory and the algorithms at a more rigorous 
level. 

The book can be used both as a reference and as a textbook for advanced 
undergraduate students and first-year graduate students in the fields of industrial 
engineering, management, operations research, computer science, mathematics, 
and other engineering disciplines that deal with the subjects of linear 
programming and network flows. Even though the book’s material requires 
some mathematical maturity, the only prerequisite is linear algebra and elemen- 
tary calculus. For the convenience of the reader, pertinent results from linear 
algebra and convex analysis are summarized in Chapter 2. 

This book can be used in several ways. It can be used in a two—course 
sequence on linear programming and network flows, in which case all of its 
material could be easily covered. The book can also be utilized in a one— 
semester course on linear programming and network flows. The instructor may 
have to omit some topics at his or her discretion. The book can also be used as a 
text for a course on either linear programming or network flows. 

Following the introductory first chapter, the second chapter presents basic 
results on linear algebra and convex analysis, along with an insightful, geomet- 
rically motivated study of the structure of polyhedral sets. The remainder of the 
book is organized into two parts: linear programming and networks flows. The 
linear programming part consists of Chapters 3 to 8. In Chapter 3 the simplex 
method is developed in detail, and in Chapter 4 the initiation of the simplex 
method by the use of artificial variables and the problem of degeneracy and 
cycling along with geometric concepts are discussed. Chapter 5 deals with some 
specializations of the simplex method and the development of optimality criteria 
in linear programming. In Chapter 6 we consider the dual problem, develop 
several computational procedures based on duality, and discuss sensitivity 
analysis (including the tolerance approach) and parametric analysis (including 
the determination of shadow prices). Chapter 7 introduces the reader to the 
decomposition principle and large-scale optimization. The equivalence of 
several decomposition techniques for linear programming problems is exhibited 
in this chapter. Chapter 8 discusses some basic computational complexity issues, 
exhibits the worst-case exponential behavior of the simplex algorithm, and 
presents Karmarkar’s polynomial-time algorithm along with a brief introduction 
to various interior point variants of this algorithm such as affine scaling 
methods, primal—dual path-following procedures, and _predictor—corrector 
techniques. These variants constitute an arsenal of computationally effective 
approaches that compare favorably with the simplex method for large, sparse, 
generally structured problems. Khachian’s polynomial-time ellipsoid algorithm 
is presented in the Exercises. 

The part on network flows consists of Chapters 9 to 12. In Chapter 9 we 
study the principal characteristics of network structured linear programming 
problems and discuss the specialization of the simplex algorithm to solve these 
problems. A detailed discussion of list structures, useful from a terminology as 
well as from an implementation viewpoint, is also presented. Chapter 10 deals 
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with the popular transportation and assignment network flow problems. 
Although the algorithmic justifications and some special techniques rely on the 
material in Chapter 9, it is possible to study Chapter 10 separately if one is sim- 
ply interested in the fundamental properties and algorithms for transportation 
and assignment problems. Chapter 11 presents the out—of-kilter algorithm along 
with some basic ingredients of primal—dual and relaxation types of algorithms 
for network flow problems. Finally, Chapter 12 covers the special topics of the 
maximal flow problem (including polynomial-time variants), the shortest path 
problem (including several efficient polynomial-time algorithms for this 
ubiquitous problem), the multicommodity minimal—cost flow problem, and a 
network synthesis or design problem. The last of these topics complements, as 
well as relies on, the techniques developed for the problems of analysis, which 
are the types of problems considered in the remainder of this book. 

In preparing revised editions of this book, we have followed two princi- 
pal objectives. Our first objective was to offer further concepts and insights into 
linear programming theory and algorithmic techniques. Toward this end, we 
have included detailed geometrically motivated discussions dealing with the 
structure of polyhedral sets, optimality conditions, and the nature of solution 
algorithms and special phenomena such as cycling. We have also added 
examples and remarks throughout the book that provide insights, and improve 
the understanding of, and correlate, the various topics discussed in the book. 
Our second objective was to update the book to the state-of-the-art while 
keeping the exposition transparent and easy to follow. In keeping with this 
spirit, several topics have now been included, such as cycling and stalling 
phenomena and their prevention (including special approaches for network flow 
problems), numerically stable implementation techniques and empirical studies 
dealing with the simplex algorithm, the tolerance approach to sensitivity 
analysis, the equivalence of the Dantzig—Wolfe decomposition, Benders’ 
partitioning method, and Lagrangian relaxation techniques for linear 
programming problems, computational complexity issues, the worst-case 
behavior of the simplex method, Khachian’s and Karmarkar’s polynomial-time 
algorithms for linear programming problems, various other interior point algo- 
rithms such as the affine scaling, primal—dual path—following, and predictor— 
corrector methods, list structures for network simplex implementations, a suc- 
cessive shortest path algorithm for linear assignment problems, polynomial— 
time scaling strategies (illustrated for the maximal flow problem), polynomial— 
time partitioned shortest path algorithms, and the network synthesis or design 
problem, among others. The writing style enables the instructor to skip several 
of these advanced topics in an undergraduate or introductory—level graduate 
course without any loss of continuity. Also, several new exercises have been 
added, including special exercises that simultaneously educate the reader on 
some related advanced material. The notes and references sections and the 
bibliography have also been updated. 

We express our gratitude once again to Dr. Jeff Kennington, Dr. Gene 
Ramsay, Dr. Ron Rardin, and Dr. Michael Todd for their many fine suggestions 
during the preparation of the first edition; to Dr. Robert N. Lehrer, former 
director of the School of Industrial and Systems Engineering at the Georgia 
Institute of Technology, for his support during the preparation of this first 
edition; to Mr. Carl H. Wohlers for preparing its bibliography; and to Mrs. Alice 
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Jarvis, Mrs. Carolyn Piersma, Miss Kaye Watkins, and Mrs. Amelia Williams 
for their typing assistance. We also thank Dr. Faiz Al-—Khayyal, Dr. Richard 
Cottle, Dr. Joanna Leleno, Dr. Craig Tovey, and Dr. Hossam Zaki among many 
others for their helpful comments in preparing this manuscript. We are grateful 
to Dr. Suleyman Tufekci, and to Drs. Joanna Leleno and Zhuangyi Liu for, 
respectively, preparing the first and second versions of the solutions manual. A 
special thanks to Dr. Barbara Fraticelli for her painstaking reading and feedback 
on the third edition of this book, and for her preparation of the solutions manual 
for the present (fourth) edition, and to Ki-Hwan Bae for his assistance with 
editing the bibliography. Finally, our deep appreciation and gratitude to Ms. 
Sandy Dalton for her magnificent single-handed feat at preparing the entire 
electronic document (including figures) of the third and fourth editions of this 
book. 


Mokhtar S. Bazaraa 
John J. Jarvis 
Hanif D. Sherali 


ONE: INTRODUCTION 


Linear programming is concerned with the optimization (minimization or 
maximization) of a linear function while satisfying a set of linear equality and/or 
inequality constraints or restrictions. The linear programming problem was first 
conceived by George B. Dantzig around 1947 while he was working as a 
mathematical advisor to the United States Air Force Comptroller on developing 
a mechanized planning tool for a time—staged deployment, training, and logistical 
supply program. Although the Soviet mathematician and economist L. V. 
Kantorovich formulated and solved a problem of this type dealing with 
organization and planning in 1939, his work remained unknown until 1959. 
Hence, the conception of the general class of linear programming problems is 
usually credited to Dantzig. Because the Air Force refers to its various plans and 
schedules to be implemented as “programs,” Dantzig’s first published paper 
addressed this problem as “Programming in a Linear Structure.” The term “linear 
programming” was actually coined by the economist and mathematician T. C. 
Koopmans in the summer of 1948 while he and Dantzig strolled near the Santa 
Monica beach in California. 

In 1949 George B. Dantzig published the ‘“‘simplex method” for solving 
linear programs. Since that time a number of individuals have contributed to the 
field of linear programming in many different ways, including theoretical 
developments, computational aspects, and exploration of new applications of the 
subject. The simplex method of linear programming enjoys wide acceptance 
because of (1) its ability to model important and complex management decision 
problems, and (2) its capability for producing solutions in a reasonable amount 
of time. In subsequent chapters of this text we shall consider the simplex method 
and its variants, with emphasis on the understanding of the methods. 

In this chapter, we introduce the linear programming problem. The 
following topics are discussed: basic definitions in linear programming, 
assumptions leading to linear models, manipulation of the problem, examples of 
linear problems, and geometric solution in the feasible region space and the 
requirement space. This chapter is elementary and may be skipped if the reader 
has previous knowledge of linear programming. 


1.1 THE LINEAR PROGRAMMING PROBLEM 


We begin our discussion by formulating a particular type of linear programming 
problem. As will be seen subsequently, any general linear programming 
problem may be manipulated into this form. 


Basic Definitions 


Consider the following linear programming problem. Here, cx; + ¢)x2 +--+ + 
CyX, is the objective function (or criterion function) to be minimized and will 
be denoted by z. The coefficients c,,cz,...,¢, are the (known) cost coefficients 
and 
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X1,X2,...,X, are the decision variables (variables, structural variables, or activity 
levels) to be determined. 


Minimize cx, +  C9X_q te + Cy Xy 
subject to Ay XY + ay2X2 edit AnXn Pa b 
AQ 4X] + Q49X2 eee Qn, > by 
: oF ie ; 
AniX1 + Am2X2 tet AnnXn 2 On 
Xy, X2, ae X, 2 O. 


The inequality U1 Fj j 2 5; denotes the ith constraint (or restriction or functional, 


structural, or technological constraint). The coefficients aij fori=1,..,m, j=1,..., 2 are 


called the technological coefficients. These technological coefficients form the constraint 
matrix A. 


ayy aq «+s Ay 
A= “21 422 eee @2n 
Ani 4m2 +++ Amn 


The column vector whose ith component is ;, which is referred to as the right- 
hand-side vector, represents the minimal requirements to be satisfied. The 
constraints x), X2,...,X, 2 0 are the nonnegativity constraints. A set of values of 
the variables x,,...,x, satisfying all the constraints is called a feasible point or a 


feasible solution. The set of all such points constitutes the feasible region or the 
feasible space. 

Using the foregoing terminology, the linear programming problem can be 
stated as follows: Among all feasible solutions, find one that minimizes (or 
maximizes) the objective function. 


Example 1.1 
Consider the following linear problem: 


Minimize 2x, + 5x 


subjectto x, + XxX 2 6 
x, = 2X > -18 
xy, X42 2 0. 


In this case, we have two decision variables x, and x2. The objective function 
to be minimized is 2x, + 5x2. The constraints and the feasible region are 


illustrated in Figure 1.1. The optimization problem is thus to find a point in the 
feasible region having the smallest possible objective value. 
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Feasible region 


Figure 1.1. Illustration of the feasible region. 


Assumptions of Linear Programming 


To represent an optimization problem as a linear program, several assumptions 
that are implicit in the linear programming formulation discussed previously are 
needed. A brief discussion of these assumptions is given next. 


1. 


Proportionality. Given a variable x;, its contribution to cost is ¢;x; 


and its contribution to the ith constraint is Aix j- This means that if x 7 


is doubled, say, so is its contribution to cost and to each of the 
constraints. To illustrate, suppose that x, is the amount of activity j 


used. For instance, if x; = 10, then the cost of this activity is 10c,. If 
x; = 20, then the cost is 20c;, and so on. This means that no savings 


(or extra costs) are realized by using more of activity /; that is, there are 
no economies or returns to scale or discounts. Also, no setup cost for 
starting the activity is realized. 

Additivity. This assumption guarantees that the total cost is the sum 
of the individual costs, and that the total contribution to the ith 
restriction is the sum of the individual contributions of the individual 
activities. In other words, there are no substitution or interaction 
effects among the activities. 

Divisibility. This assumption ensures that the decision variables can 
be divided into any fractional levels so that non-integral values for 
the decision variables are permitted. 


Deterministic. The coefficients c pp ye and 5; are all known 


deterministically. Any probabilistic or stochastic elements inherent in 
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demands, costs, prices, resource availabilities, usages, and so on are all 
assumed to be approximated by these coefficients through some 
deterministic equivalent. 


It is important to recognize that if a linear programming problem is being 
used to model a given situation, then the aforementioned assumptions are 
implied to hold, at least over some anticipated operating range for the activities. 
When Dantzig first presented his linear programming model to a meeting of the 
Econometric Society in Wisconsin, the famous economist H. Hotelling critically 
remarked that in reality, the world is indeed nonlinear. As Dantzig recounts, the 
well-known mathematician John von Neumann came to his rescue by counter- 
ing that the talk was about “Linear” Programming and was based on a set of 
postulated axioms. Quite simply, a user may apply this technique if and only if 
the application fits the stated axioms. 

Despite the seemingly restrictive assumptions, linear programs are among 
the most widely used models today. They represent several systems quite satis- 
factorily, and they are capable of providing a large amount of information 
besides simply a solution, as we shall see later, particularly in Chapter 6. 
Moreover, they are also often used to solve certain types of nonlinear 
optimization problems via (successive) linear approximations and constitute an 
important tool in solution methods for linear discrete optimization problems 
having integer-restricted variables. 


Problem Manipulation 


Recall that a linear program is a problem of minimizing or maximizing a linear 
function in the presence of linear inequality and/or equality constraints. By 
simple manipulations the problem can be transformed from one form to another 
equivalent form. These manipulations are most useful in linear programming, as 
will be seen throughout the text. 


INEQUALITIES AND EQUATIONS 

An inequality can be easily transformed into an equation. To illustrate, consider the 
constraint given by a1 WyX pe b;. This constraint can be put in an equation form 
by subtracting the nonnegative surplus or slack variable x,,,; (sometimes denoted by 
5;) leading to 2} _)4j%; — Xn4; = 5; and x,4; 2 0. Similarly, the constraint 


n } j n 
Dial GX; < b; is equivalent to D1 GX) + x, 


i*j n+i = 5; and x,,; 2 0. Also, an 


nti 


equation of the form a1 yX '; = 6; can be transformed into the two inequalities 


wi a1 7 <5; and Da Gy ; 2 5;, although this is not the practice. 


NONNEGATIVITY OF THE VARIABLES 


For most practical problems the variables represent physical quantities, and hence 
must be nonnegative. The simplex method is designed to solve linear programs 
where the variables are nonnegative. If a variable x; is unrestricted in sign, then it 


can be replaced by xj - x} where xj > 0 and a 2 0. If »,...,x, are some k 
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variables that are all unrestricted in sign, then only one additional variable x” is 


needed in the equivalent transformation: x j= xj —x" for j = 1, .., k, where 


xj 20 forj=1,...,£, and x” 2 0. (Here, —x” plays the role of representing the 
most negative variable, while all the other variables x j are x; above this value.) 
Alternatively, one could solve for each unrestricted variable in terms of the other 
variables using any equation in which it appears, eliminate this variable from the 
problem by substitution using this equation, and then discard this equation from 
the problem. However, this strategy is seldom used from a data management and 
numerical implementation viewpoint. Continuing, if x; 2 4;, then the new 


variable xj =x; — £; is automatically nonnegative. Also, if a variable x; is 


restricted such that x; < u;, where we might possibly have u; < 0, then the 


substitution x’; = u; —x, produces a nonnegative variable x’. 


MINIMIZATION AND MAXIMIZATION PROBLEMS 


Another problem manipulation is to convert a maximization problem into a 
minimization problem and conversely. Note that over any region, 


n n 
maximum > c jx; = —minimum > — €;X;. 

j=l j=l 
Hence, a maximization (minimization) problem can be converted into a minimi- 
zation (maximization) problem by multiplying the coefficients of the objective 
function by —1. After the optimization of the new problem is completed, the 
objective value of the old problem is --1 times the optimal objective value of the 
new problem. 


Standard and Canonical Formats 


From the foregoing discussion, we have seen that any given linear program can 
be put in different equivalent forms by suitable manipulations. In particular, two 
forms will be useful. These are the standard and the canonical forms. A linear 
program is said to be in standard format if all restrictions are equalities and all 
variables are nonnegative. The simplex method is designed to be applied only 
after the problem is put in standard form. The canonical form is also useful, 
especially in exploiting duality relationships. A minimization problem is in 
canonical form if all variables are nonnegative and all the constraints are of the 
> type. A maximization problem is in canonical form if all the variables are 
nonnegative and all the constraints are of the < type. The standard and canonical 
forms are summarized in Table 1.1. 


Linear Programming in Matrix Notation 


A linear programming problem can be stated in a more convenient form using 
matrix notation. To illustrate, consider the following problem: 
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n 
Minimize x CjX; 
j=l 
. n 
subjectto > ayx; = b, i=l,...,m 
j=l 
xj 2 0, jH=l,..,n. 
Denote the row vector (c,,C,....C,) by ¢, and consider the following column 


vectors x and b, and the m x n matrix A. 


x b, 4, a2... Ay 
x=/2] pal] aa] 92 + Gn 
Xn bn QGm1 4m2 Qn 


Then the problem can be written as follows: 


Minimize cx 
subject to Ax 
x 


b 
0. 


Vv il 


The problem can also be conveniently represented via the columns of A. 
Denoting A by [aj,a,...,a,,] where a j is the jth column of A, the problem can 


be formulated as follows: 


n 
Minimize > C jx; 
j=l 


n 
subjectto Yaj;x; = b 
j=l 


IV 


xj 


1.2 LINEAR PROGRAMMING MODELING AND EXAMPLES 


0, jH=ly...n. 


The modeling and analysis of an operations research problem in general, and a 
linear programming problem in particular, evolves through several stages. The 
problem formulation phase involves a detailed study of the system, data 
collection, and the identification of the specific problem that needs to be 
analyzed (often the encapsulated problem may only be part of an overall system 
problem), along with the system constraints, restrictions, or limitations, and the 
objective function(s). Note that in real-world contexts, there frequently already 
exists an operating solution and it is usually advisable to preserve a degree of 
persistency with respect to this solution, i.e., to limit changes from it (e.g., to 
limit the number of price changes, or decision option modifications, or changes 
in percentage resource consumptions, or to limit changing some entity 
contingent on changing another related entity). Such issues, aside from 
technological or structural aspects of the problem, should also be modeled into 
the problem constraints. 

The next stage involves the construction of an abstraction or an 
idealization of the problem through a mathematical model. Care must be taken 
to ensure that the model satisfactorily represents the system being analyzed, 
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while keeping the model mathematically tractable. This compromise must be 
made judiciously, and the underlying assumptions inherent in the model must be 
properly considered. It must be borne in mind that from this point onward, the 
solutions obtained will be solutions to the model and not necessarily solutions to 
the actual system unless the model adequately represents the true situation. 

The third step is to derive a solution. A proper technique that exploits any 
special structures (if present) must be chosen or designed. One or more optimal 
solutions may be sought, or only a heuristic or an approximate solution may be 
determined along with some assessment of its quality. In the case of multiple 
objective functions, one may seek efficient or Pareto—optimal solutions, that is, 
solutions that are such that a further improvement in any objective function 
value is necessarily accompanied by a detriment in some other objective 
function value. 

The fourth stage is model testing, analysis, and (possibly) restructuring. 
One examines the model solution and its sensitivity to relevant system 
parameters, and studies its predictions to various what-if types of scenarios. 
This analysis provides insights into the system. One can also use this analysis to 
ascertain the reliability of the model by comparing the predicted outcomes with 
the expected outcomes, using either past experience or conducting this test 
retroactively using historical data. At this stage, one may wish to enrich the 
model further by incorporating other important features of the system that have 
not been modeled as yet, or, on the other hand, one may choose to simplify the 
model. 

The final stage is implementation. The primary purpose of a model is to 
interactively aid in the decision-making process. The model should never 
replace the decision maker. Often a “frank—factor” based on judgment and 
experience needs to be applied to the model solution before making policy 
decisions. Also, a model should be treated as a “living” entity that needs to be 
nurtured over time, i.e., model parameters, assumptions, and restrictions should 
be periodically revisited in order to keep the model current, relevant, and valid. 

We describe several problems that can be formulated as linear programs. 
The purpose is to exhibit the varieties of problems that can be recognized and 
expressed in precise mathematical terms as linear programs. 


Feed Mix Problem 


An agricultural mill manufactures feed for chickens. This is done by mixing 
several ingredients, such as corn, limestone, or alfalfa. The mixing is to be done in 
such a way that the feed meets certain levels for different types of nutrients, such 
as protein, calcium, carbohydrates, and vitamins. To be more specific, suppose 
that n ingredients j = 1,..., 7 and m nutrients / = 1,..., # are considered. Let the unit 
cost of ingredient j be c; and let the amount of ingredient j to be used be x;. The 


cost is therefore wraie ';;- If the amount of the final product needed is 5, then 


we must have Lax p= b. Further suppose that ay is the amount of nutrient i 


present in a unit of ingredient j, and that the acceptable lower and upper limits of 
nutrient i in a unit of the chicken feed are £, and u;, respectively. Therefore, we 
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must have the constraints £4b < Lay) <ujb for i = 1,.., m. Finally, 


because of shortages, suppose that the mill cannot acquire more than wu, units of 


J 
ingredient 7. The problem of mixing the ingredients such that the cost is 
minimized and the restrictions are met can be formulated as follows: 


Minimize GX +  €2%2 eet Onn 

subject to Xo 4 Xp fee KF = b 
bey Sam + 42%2 t+ nn <u 

bly SayX + 492X240  AanXn  <  OUY 

: : : 

ben SyyX + Am2XQ2_ e+ GX < buy, 

OSsxy < Wy 

O0< 2 < ud 
OSX, < Uy. 


Production Scheduling: An Optimal Control Problem 


A company wishes to determine the production rate over the planning 
horizon of the next T weeks such that the known demand is satisfied and the 
total production and inventory cost is minimized. Let the known demand rate at 
time ¢ be g(t), and similarly, denote the production rate and inventory at time ¢ 
by x(t) and y(t), respectively. Furthermore, suppose that the initial inventory at 
time 0 is yo and that the desired inventory at the end of the planning horizon is 


yr. Suppose that the inventory cost is proportional to the units in storage, so 
that the inventory cost is given by c, fe y(t) dt where c,; > 0 is known. Also, 
suppose that the production cost is proportional to the rate of production, and is 
therefore given by CA x(t) dt. Then the total cost is i [o.y(t) + ¢yx(t)] dt. 
Also note that the inventory at any time is given according to the relationship 


y(t) = yq + fplx(c) - g(z)] dr, fort € [0,7]. 


Suppose that no backlogs are allowed; that is, all demand must be satisfied. 
Furthermore, suppose that the present manufacturing capacity restricts the 
production rate so that it does not exceed 5, at any time. Also, assume that the 


available storage restricts the maximum inventory to be less than or equal to dy. 
Hence, the production scheduling problem can be stated as follows: 


Minimize [9 [¢y(t) + cyx(1)]at 


subject to y(t)= yo + [hlx(c)-g(2)]dr, for te[0,7] 


WT) = yr 
Osx <h, for te[0,7] 
0< y(t) S by, for te[0,7)]. 


The foregoing model is a linear control problem, where the control variable is 
the production rate x(t) and the state variable is the inventory level y(t). The 
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problem can be approximated by a linear program by discretizing the continuous 
variables x and y. First, the planning horizon [0, 7] is divided into n smaller 


periods [0,A],[A,2A],...,[(” — l)A,nA], where nA = T. The production rate, 


the inventory, and the demand rate are assumed constant over each period. In 
particular, let the production rate, the inventory, and the demand rate in period j 
be x;, y,, and g;, respectively. Then, the production scheduling problem can 


be approximated by the following linear program (why?). 
n n 
Minimize > (c\A)y; +2 (cpA)x; 
j=l j=l 


subject to V;,=Vjat; ~g))A, jal.m 


Yn =J)r 
O<x; <b, J=l,...5n 
O<y; <b, J=,...5M. 


Cutting Stock Problem 


A manufacturer of metal sheets produces rolls of standard fixed width w and of 
standard length ¢. A large order is placed by a customer who needs sheets of 
width w and varying lengths. In particular, 5; sheets with length 2; and width w 
for i= 1,..., m are ordered. The manufacturer would like to cut the standard rolls in 
such a way as to satisfy the order and to minimize the waste. Because scrap pieces 
are useless to the manufacturer, the objective is to minimize the number of rolls 
needed to satisfy the order. Given a standard sheet of length @, there are many 
ways of cutting it. Each such way is called a cutting pattern. The jth cutting 
pattern is characterized by the column vector a; where the ith component of a ,, 


namely a,;, is a nonnegative integer denoting the number of sheets of length £; in 


the jth pattern. For instance, suppose that the standard sheets have length @ = 10 
meters and that sheets of lengths 1.5, 2.5, 3.0, and 4.0 meters are needed. The 
following are typical cutting patterns: 


3 0 
2 4 

a = oP a = oP a3 = 
0 0 


Note that the vector a; represents a cutting pattern if and only if D)7_, 4,2; S ¢ 
and each a, is a nonnegative integer. The number of cutting patterns n is finite. 
If we let x; be the number of standard rolls cut according to the jth pattern, the 


problem can be formulated as follows: 
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n 
Minimize > x 7 
j=l 


n 

subject to 2 ayjXj = b;, i=1,...,.m 
j=l 

xj > 0, J=l,..n 

x; integer, JH=l,...5n. 


If the integrality requirement on the x '; ~variables is dropped, the problem is a 


linear program. Of course, the difficulty with this problem is that the number of 
possible cutting patterns n is very large, and also, it is not computationally 
feasible to enumerate each cutting pattern and its column a ; beforehand. The 


decomposition algorithm of Chapter 7 is particularly suited to solve this 
problem, where a new cutting pattern is generated at each iteration (see also 
Exercise 7.28). In Section 6.7 we suggest a method for handling the integrality 
requirements. 


The Transportation Problem 


The Brazilian coffee company processes coffee beans into coffee at m plants. 
The coffee is then shipped every week to n warehouses in major cities for retail, 
distribution, and exporting. Suppose that the unit shipping cost from plant i to 
warehouse j is cj. Furthermore, suppose that the production capacity at plant i 
is a; and that the demand at warehouse j is 6,. It is desired to find the 
production-shipping pattern x, from plant i to warehouse j, i = 1,..., m,j = 1,..., 
n, which minimizes the overall shipping cost. This is the well-known 
transportation problem. The essential elements of the problem are shown in the 
network of Figure 1.2. The transportation problem can be formulated as the 
following linear program: 


m n 
Minimize 2 2 CyXij 
i=l j= 


n 
subject to DX xy Sj; i=l,...,m 
j=l 
m 
2 iy = Oj JH." 
{= 
xy 29, i=l.jm, j=ly.in 


Capital Budgeting Problem 


A municipal construction project has funding requirements over the next four 
years of $2 million, $4 million, $8 million, and $5 million, respectively. Assume 
that all of the money for a given year is required at the beginning of the year. 
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Plants Warehouses 


Figure 1.2. The transportation problem. 


The city intends to sell exactly enough long-term bonds to cover the project 
funding requirements, and all of these bonds, regardless of when they are sold, 
will be paid off (mature) on the same date in a distant future year. The long— 
term bond market interest rates (that is, the costs of selling bonds) for the next 
four years are projected to be 7 percent, 6 percent, 6.5 percent, and 7.5 percent, 
respectively. Bond interest paid will commence one year after the project is 
complete and will continue for 20 years, after which the bonds will be paid off. 
During the same period, the short-term interest rates on time deposits (that is, 
what the city can earn on deposits) are projected to be 6 percent, 5.5 percent, 
and 4.5 percent, respectively (the city will clearly not invest money in short- 
term deposits during the fourth year). What is the city’s optimal strategy for 
selling bonds and depositing funds in time accounts in order to complete the 
construction project? 


To formulate this problem as a linear program, let x J = 1,...,4, be the amount 


of bonds sold at the beginning of each year 7. When bonds are sold, some of the 
money will immediately be used for construction and some money will be 
placed in short-term deposits to be used in later years. Let y;, j = l,....3, be the 


money placed in time deposits at the beginning of year 7. Consider the 
beginning of the first year. The amount of bonds sold minus the amount of time 
deposits made will be used for the funding requirement at that year. Thus, we 
may write 

my - y= 2. 


We could have expressed this constraint as >. However, it is clear in this case 
that any excess funds will be deposited so that equality is also acceptable. 

Consider the beginning of the second year. In addition to bonds sold and 
time deposits made, we also have time deposits plus interest becoming available 
from the previous year. Thus, we have 


1.06), at XX. — ¥2 = 4. 


Introduction 13 


The third and fourth constraints are constructed in a similar manner. 

Ignoring the fact that the amounts occur in different years (that is, the 
time value of money), the unit cost of selling bonds is 20 times the interest rate. 
Thus, for bonds sold at the beginning of the first year we have c, = 20(0.07). 


The other cost coefficients are computed similarly. 
Accordingly, the linear programming model is given as follows: 


Minimize 20(0.07) x, + 20(0.06) x. + 20(0.065) x, + 20(0.075) x4 
subject to x, — yy 7 
1.06, + x. - y2 z 
1.055 y,+ x,- 3 = 
1.045 y3 + x4 =5 


xX, x2; X3, X4> Yi» Y2> V3 2 0. 


Tanker Scheduling Problem 


A shipline company requires a fleet of ships to service requirements for carrying 
cargo between six cities. There are four specific routes that must be served 
daily. These routes and the number of ships required for each route are as 
follows: 


NUMBER OF SHIPS 


ROUTE # ORIGIN DESTINATION PER DAY NEEDED 
1 Dhahran New York 3 
2 Marseilles Istanbul 2 
3 Naples Mumbai 1 
4 New York Marseilles 1 


All cargo is compatible, and therefore only one type of ship is needed. The 
travel time matrix between the various cities is shown. 


iq 
Se ee ~ ee 4 
nN fz 3 Oo Ps = 
5 n Zz a = aa 
Se © < eB < 2 
< << & gy an =) 
Ze 822 A 2 
Naples 0 1 2 414 7 7 
Marseilles 1 0 3. «13 8 8 
Istanbul 2 3 0 15 5 5 te matrix (days) 
New York 14 13 15 0 17 20 7 
Dhahran 7 8 5 17 0 3 
Mumbai 7 8 5 20 3 0 


It takes one day to off-load and one day to on—load each ship. How many ships 
must the shipline company purchase? 
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In addition to nonnegativity restrictions, there are two types of constraints 
that must be maintained in this problem. First, we must ensure that ships coming 
off of some route get assigned to some (other) route. Second, we must ensure 
that each route gets its required number of ships per day. Let xy be the number 


of ships per day coming off of route i and assigned to route j. Let b; represent 


the number of ships per day required on route /. 
To ensure that ships from a given route get assigned to other routes, we 
write the constraint 


j=l 
To ensure that a given route gets its required number of ships, we write the 
constraint 


4 
> XxX = b;, i= 1,...,4. 
k=! 


Computing the cost coefficients is a bit more involved. Since the objective is to 
minimize the total number of ships, let c; be the number of ships required to 


ensure a continuous daily flow of one ship coming off of route 7 and assigned to 


route j. To illustrate the computation of these c,; -coefficients, consider c23. It 


takes one day to load a ship at Marseilles, three days to travel from Marseilles to 
Istanbul, one day to unload cargo at Istanbul, and two days to head from 
Istanbul to Naples—a total of seven days. This implies that seven ships are 
needed to ensure that one ship will be assigned daily from route 2 to route 3 
(why?). In particular, one ship will be on—loading at Marseilles, three ships en 
route from Marseilles to Istanbul, one ship off-loading at Istanbul, and two 
ships en route from Istanbul to Naples. 

In general, cj is given as follows: 

cy = one day for on—loading + number of days for transit on route i 
one day for off-loading 
number of days for travel from the destination of route / to the 
origin of route /. 


+ 
+ 


Therefore, the tanker scheduling problem can be formulated as follows: 


Minimize 36x11 + 32x12 “fe 33x13 + 19x14 + 10x, + 8X29 + 7X73 
+20X4 + 12x3) + 17x32 + 16x33 + 29x34 + 23X41 
+15x42 + 16x43 + 28X44 


4 
subject to > xij = b; i= 1, 2; 3,4 
j=l 
4 
LX = 8, i=1,2,3,4 
k=! 
xX; 2 0, i,j= 1,2, 3,4, 


y 
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where b, =3, b) =2, b; =1, and by = 1. 

It can be easily seen that this is another application of the transportation 
problem (it is instructive for the reader to form the origins and destinations of 
the corresponding transportation problem). 


Multiperiod Coal Blending and Distribution Problem 


A southwest Virginia coal company owns several mines that produce coal at 
different given rates, and having known quality (ash and sulfur content) 
specifications that vary over mines as well as over time periods at each mine. 
This coal needs to be shipped to silo facilities where it can be possibly subjected 
to a beneficiation (cleaning) process, in order to partially reduce its ash and 
sulfur content to a desired degree. The different grades of coal then need to be 
blended at individual silo facilities before being shipped to customers in order to 
satisfy demands for various quantities having stipulated quality specifications. 
The aim is to determine optimal schedules over a multiperiod time horizon for 
shipping coal from mines to silos, cleaning and blending the coal at the silos, 
and distributing the coal to the customers, subject to production capacity, 
storage, material flow balance, shipment, and quality requirement restrictions, 
so as to satisfy the demand at a minimum total cost, including revenues due to 
rebates for possibly shipping coal to customers that is of a better quality than the 
minimum acceptable specified level. 

Suppose that this problem involves i = 1,..., m mines, j = 1,..., J silos, k = 
1,..., K customers, and that we are considering ¢ = 1,..., T (= 3) time periods. Let 
Dj be the production (in tons of coal) at mine i during period ¢, and let a,, and 


Si, Tespectively denote the ash and sulfur percentage content in the coal 
produced at mine i during period ¢. Any excess coal not shipped must be stored 


at the site of the mine at a per—period storage cost of on per ton at mine i, 
where the capacity of the storage facility at mine iis given by M;. 


Let A, denote the permissible flow transfer arcs (i, 7) from mine i to silo 
J, and let F} = {7 : (if) € 4} and Ri = {i: (i,j) € A}. The transportation 
cost per ton from mine i to silo / is denoted by Cis for each (i,j) € 4). Each 


silo j has a storage capacity of S;, and a per—ton storage cost of oe per period. 
Assume that at the beginning of the time horizon, there exists an initial amount 
of q) tons of coal stored at silo /, having an ash and sulfur percentage content of 


0 0 
aj and Sis 


cleaning facilities, where any coal coming from mine i to such a silo / is cleaned 


respectively. Some of the silos are equipped with beneficiation or 


at a cost of op per ton, resulting in the ash and sulfur content being respectively 
attenuated by a factor of f € (0,1] and yj € (0,1], and the total weight being 


thereby attenuated by a factor of a, € (0,1) (hence, for one ton input, the 
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output is a tons, which is then stored for shipment). Note that for silos that do 


not have any cleaning facilities, we assume that cf =O,and a, =f; = 77 = 1. 


Let A, denote the feasible flow transfer arcs (j, k) from silo 7 to customer 
k,and let F? = {k : (j,k) © 4p}, and Rp = {7 : (j,k) © Ay}. The transport- 
tation cost per ton from silo j to customer & is denoted by cf , for each (j,k) € 
Ay. Additionally, if ¢, is the time period for a certain mine to silo shipment 
(assumed to occur at the beginning of the period), and f is the time period for a 
continuing silo to customer shipment (assumed to occur at the end of the 
period), then the shipment lag between the two coal flows is given by tj —t. A 
maximum of a three-period shipment lag is permitted between the coal 
production at any mine and its ultimate shipment to customers through any silo, 


based on an estimate of the maximum clearance time at the silos. (Actual 
shipment times from mines to silos are assumed to be negligible.) The demand 


placed (in tons of coal) by customer & during period ¢ is given by d;,,, with ash 
and sulfur percentage contents being respectively required to lie in the intervals 
defined by the lower and upper limits [¢4,, ug, ] and [¢%,,uj,]. There is also a 
revenue earned of ~, per—ton per—percentage point that falls below the maxi— 


mum specified percentage uj, of ash content in the coal delivered to customer k 


during period ¢. 
To model this problem, we first define a set of principal decision 


variables as vie = amount (tons) of coal shipped from mine / to silo j in period 


t, with continued shipment to customer é in period r (where zt = ¢, t + 1, t + 2, 


based on the three—period shipment lag restriction), and Vue = amount (tons) 


of coal that is in initial storage at silo 7, which is shipped to customer & in period 
t (where t = 1,2,3, based on a three period dissipation limit). Controlled by 
these principal decisions, there are four other auxiliary decision variables 


defined as follows: xf = slack variable that represents the amount (tons) of coal 


remaining in storage at mine i during period 6; x5 = accumulated storage 


amount (tons) of coal in silo j during period 5; z¢, = percentage ash content in 


the blended coal that is ultimately delivered to customer k in period 7 and zi, = 


percentage sulfur content in the blended coal that is ultimately delivered to 
customer & in period rt. The linear programming model is then given as follows, 
where the objective function records the transportation, cleaning, and storage 
costs, along with the revenue term over the horizon 1,...,7 of interest. The 
respective sets of constraints represent the flow balance at the mines, storage 
capacity restrictions at the mines, flow balance at the silos, storage capacity 
restrictions at the silos, the dissipation of the initial storage at the silos, the 
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demand satisfaction constraints, the ash content identities, the quality bound 
specifications with respect to the ash content, the sulfur content identities, the 
quality bound specifications with respect to the sulfur content, and the 
remaining logical nonnegativity restrictions. (All undefined variables and 
summation terms are assumed to be zero. Also, see Exercises 1.19-1.21.) 


T min{t+2,7} 1 B 2 
Minimize x > FY > [cy + cy + Cie 
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1.3 GEOMETRIC SOLUTION 


We describe here a geometric procedure for solving a linear programming 
problem. Even though this method is only suitable for very small problems, it 
provides a great deal of insight into the linear programming problem. To be 
more specific, consider the following problem: 


Minimize ex 
subject to Ax 


oo 


Note that the feasible region consists of all vectors x satisfying Ax = b and x > 
0. Among all such points, we wish to find a point having a minimal value of ex. 
Note that points having the same objective value z satisfy the equation ex = z, 


that is, Li a1cj*/ = z. Since z is to be minimized, then the plane (line in a 
two-dimensional space) Laie ‘jj = Zz must be moved parallel to itself in the 


direction that minimizes the objective the most. This direction is —c, and hence 
the plane is moved in the direction of —¢c as much as possible, while maintaining 
contact with the feasible region. This process is illustrated in Figure 1.3. Note 


that as the optimal point x" is reached, the line ¢,x, + ¢)x7 = z", where z* = 


cyx,_ + x}, cannot be moved farther in the direction —e = (—c¢,,-c>), because 
this will only lead to points outside the feasible region. In other words, one 


cannot move from x* ina direction that makes an acute angle with -c,1.e., a 
direction that reduces the objective function value, while remaining feasible. We 


therefore conclude that x” is indeed an optimal solution. Needless to say, for a 
maximization problem, the plane cx = z must be moved as much as possible in 
the direction ¢, while maintaining contact with the feasible region. 

The foregoing process is convenient for problems having two variables 
and is obviously impractical for problems with more than three variables. It is 


worth noting that the optimal point x” in Figure 1.3 is one of the five corner 
points that are called extreme points. We shall show in Section 3.1 that if a 
linear program in standard or canonical form has a finite optimal solution, then 
it has an optimal corner (or extreme) point solution. 
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Objective 
decreases 
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Cy + CoX2Q = 29,27 <2] 
CyX + C2X2 = 2] 


Figure 1.3. Geometric solution. 


Example 1.2 


Minimize -x, — 3x, 

subjectto x + Xx, < 6 
—x) + 2X9 < 8 
X15 x2 > 0 


The feasible region is illustrated in Figure 1.4. For example, consider the 
second constraint. The equation associated with this constraint is —x, + 2x) = 


8. The gradient or the partial derivative vector of the linear function —x, + 2x, 


is ea Hence, —x, + 2x2 increases in any direction making an acute angle 


with Ht and decreases in any direction making an acute angle EF Conse- 


quently, the region feasible to —x,; + 2x) < 8 relative to the equation —x, + 
2x2 = 8 is as shown in Figure 1.4 and encompasses points having decreasing 
values of —x, + 2x, from the value 8. (Alternatively, this region may be 
determined relative to the equation —x, + 2x =8 by testing the feasibility of a 
point, for example, the origin.) Similarly, the region feasible to the first 
constraint is as shown. (Try adding the constraint —2x, + 3x, > 0 to this 
figure.) The nonnegativity constraints restrict the points to be in the first 
quadrant. The equations —x, — 3x2 = z, for different values of z, are called the 
objective contours and are represented by dotted lines in Figure 1.4. In particular 
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Figure 1.4. Numerical example. 


the contour —x, — 3x. = z = 0 passes through the origin. We move onto lower 


valued contours in the direction —e = (1, 3) as much as possible until the 
optimal point (4/3, 14/3) is reached. 

In this example we had a unique optimal solution. Other cases may occur 
depending on the problem structure. All possible cases that may arise are 
summarized below (for a minimization problem). 


1. 


Unique Optimal Solution. \f the optimal solution is unique, then it 
occurs at an extreme point. Figures 1.5a and b show a unique optimal 
solution. In Figure 1.5a the feasible region is bounded; that is, there 
is a ball of finite radius centered at, say, the origin that contains the 
feasible region. In Figure |.5b the feasible region is not bounded. In 
each case, however, a finite unique optimal solution is obtained. 

Alternative Optimal Solutions. This case is illustrated in Figure 1.6. 
Note that in Figure |.6a the feasible region is bounded. The two 


corner points x; and x> are optimal, as well as any point on the line 


segment joining them. In Figure 1.6b the feasible region is 
unbounded but the optimal objective is finite. Any point on the “ray” 


with vertex x" in Figure 1.6b is optimal. Hence, the optimal solution 
Set is unbounded. 


In both cases (1) and (2), it is instructive to make the following 


observation. Pick an optimal solution x” in Figure 1.5 or 1.6, corner 
point or not. Draw the normal vectors to the constraints passing 


through x” pointing in the outward direction with respect to the 


feasible region. Also, construct the vector —c at x’. Note that the 
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“cone” spanned by the normals to the constraints passing through x” 
contains the vector —c. This is in fact the necessary and sufficient 


condition for x” to be optimal, and will be formally established 
later. Intuitively, when this condition occurs, we can see that there is 
no direction along which a motion is possible that would improve the 
objective function while remaining feasible. Such a direction would 
have to make an acute angle with —c to improve the objective value 


and simultaneously make an angle >90° with respect to each of the 


normals to the constraints passing through x” in order to maintain 
feasibility for some step length along this direction. This is 
impossible at any optimal solution, although it is possible at any 
nonoptimal solution. 

3. Unbounded Optimal Objective Value. This case is illustrated in 
Figure 1.7 where both the feasible region and the optimal objective 
value are unbounded. For a minimization problem, the plane ex = z 
can be moved in the direction —e indefinitely while always inter— 
secting with the feasible region. In this case, the optimal objective 
value is unbounded (with value —co) and no optimal solution exists. 


Examining Figure 1.8, it is clear that there exists no point (x),x7) satisfying 


these inequalities. The problem is said to be infeasible, inconsistent, or with an 
empty feasible region. Again, we say that no optimal solution exists in this case. 


Unique f 

optimal - 
‘ - 

solution, ¢ ’ 


(a) 
Figure 1.5. Unique optimal solution: (a) Bounded region. (b) Unbounded region. 
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» Alternative 
~,X1 optimal solutions 


(b) 
Figure 1.6. Alternative optima: (a) Bounded region. (b) Unbounded region. 


Objective contours can 

be moved indefinitely 

in the direction -e while 
intersecting with the feasible 
region 


Figure 1.7. Unbounded optimal] objective value. 


4. Empty Feasible Region. In this case, the system of equations and/or 
inequalities defining the feasible region is inconsistent. To illustrate, 
consider the following problem: 


Minimize -2x, + 3x, 


subjectto -x, + 2x, < 2 
2x = X4 < 3 

xX4 => 4 

X]; X49 > 0. 


1.4 THE REQUIREMENT SPACE 


The linear programming problem can be interpreted and solved geometrically in 
another space, referred to as the requirement space. 
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Figure 1.8. An example of an empty feasible region. 


Interpretation of Feasibility 


Consider the following linear programming problem in standard form: 
Minimize ex 
subjectto Ax = b 
x >= 0, 
where A is an m xn matrix whose jth column is denoted by a;. The problem 


can be rewritten as follows: 


n 
Minimize > c 


xX; 
J 
J=l 


J 
F n 

subject to )) a,x; = b 
j=l 


x; 2 0, j= hans 


Given the vectors a;,a2,...,a,,, we wish to find nonnegative scalars x, x7, Xp, 


such that Li-18 py =b and such that viae ';%; is minimized. Note, however, 


that the collection of vectors of the form ¥\"_,a;x,;, where x,, x5,...,x, > 0, is 
pol airy 12 X2.-9Xpy , 


the cone generated by aj,a9,...,a, (see Figure 1.9). Thus, the problem has a 


feasible solution if and only if the vector b belongs to this cone. Since the vector b 
usually reflects requirements to be satisfied, Figure 1.9 is referred to as illustrating 
the requirement space. 
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(a) (b) 
Figure 1.9. Interpretation of feasibility in the requirement space: (a) 
Feasible region is not empty. (b) Feasible region is empty. 


(a) (b) 


Figure 1.10. Illustration of the requirement space: (a) System 1 is feasible. 
(b) System 2 is inconsistent. 
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Example 1.3 


Consider the following two systems: 


System 1: 
2x) + . +t *% = 2 
—X} + 3x9 + xy = 3 
X, x9, X3, x4 2 0. 
System 2: 
2x) + xX. + X3 = -l 
—X) + 3x, + Xy = 2 
X1, x9, X35 Xx, 2 0. 


Figure 1.10 shows the requirement space of both systems. For System 1, the 


vector b belongs to the cone generated by the vectors ET 3} lo} and 


a and hence admits feasible solutions. For the second system, b does not 


belong to the corresponding cone and the system is then inconsistent. 


The Requirement Space and Inequality Constraints 


We now illustrate the interpretation of feasibility for the inequality case. 
Consider the following inequality system: 


xj >0, jHly.uin. 


Note that the collection of vectors i =18 jx; , where xj, 2 0 for j = 1,...,7, is 


the cone generated by a),4),...,a,. Ifa feasible solution exists, then this cone 


must intersect the collection of vectors that are less than or equal to the 
requirement vector b. Figure 1.11 shows both a feasible system and an 
infeasible system. 

Optimality 

We have seen that the system 18 jx; = b and x; 2 0 for j = 1,..., 7 is 
feasible if and only if b belongs to the cone generated by a),49,...,a,. The 


variables x,,X2,....X, must be chosen so that feasibility is satisfied and 
vial j%j 18 minimized. Therefore, the linear programming problem can be 


stated as follows. Find nonnegative x), x3,...,x,, such that 


Lo fs + [2 fe ++ [Se es = [iB 
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Collection of Collection of 
vectors = b vectors <b 
(a) (b) 


Figure 1.11. Requirement space and inequality constraints: (a) System is 
feasible. (b) System is infeasible. 


where the objective z is to be minimized. In other words we seek to represent the 


vector El for the smallest possible z, in the cone spanned by the vectors 


E | E | |< i The reader should note that the price we must pay for 
ay a2 an 


including the objective function explicitly in the requirement space is to increase 
the dimensionality from m to m+ 1. 


Example 1.4 
Minimize -2x, —- 3x 
subject to x + 2x, < 2 
XY, X92 > 0. 


Add the slack variable x3; 2 0. The problem is then to choose x, x2, x3 20 


such that 
—2 -3 0 
[ils +[a)s +[t}s = [3 
where z is to be minimized. The cone generated by the vectors Pat ier and 


Hi is shown in Figure 1.12. We want to choose a vector [3 in this cone 


having a minimal value for z. This gives the optimal solution z* = —4 with 


x; = 2 (the multiplier associated with iD and x7 = x3 = 0. 
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Example 1.5 


Minimize -2x, —- 3x 
subject to x + 2x 2 
x, x2 > 0. 


Obviously the optimal objective value is unbounded. We illustrate this fact in 
the requirement space. Subtracting the slack (or surplus) variable x3 = 0, the 


problem can be restated as follows: Find x, x2, x3; 2 0 such that 
-2 -3 0 
[ils +[2}2 + [aps =[3} 


and such that z is minimized. The cone generated by 14}. Et and Hi is 


shown in Figure 1.13. We want to choose H in this cone having the smallest 


Zz 
2 


having an arbitrarily small value for z. Therefore, the objective value z can be 
driven to —0o, or it is unbounded. 


possible value for z. Note that we can find points of the form in the cone 


1.5 NOTATION 


Throughout the text, we shall utilize notation that is, insofar as possible, 
consistent with generally accepted standards for the fields of mathematics and 
operations research. In this section, we indicate some of the notation that may 
require special attention, either because of its infrequency of use in the linear 
programming literature, or else because of the possibility of confusion with 
other terms. 


Minimal 7 


value z” = —4 


Figure 1.12. Optimal objective value in the requirement space. 
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Points of the form 


Figure 1.13. Unbounded optimal objective value in the requirement space. 


In Chapter 2, we shall review material on vectors and matrices. We 
indicate vectors by lowercase, boldface Greek or Roman letters or numerals, 
such as a, b, x, 1, A; matrices by uppercase, boldface Greek or Roman letters, 
such as A, B, N, ®; and all scalars by Greek or Roman letters or numerals that 
are not boldface, such as a, b, 1, €. Column vectors are generally denoted by 
subscripts, such as a,, unless clear in the context. When special emphasis is 


required, row vectors are indicated by superscripts, such as a’. A superscript ¢ 
will denote the transpose operation. 

In calculus, the partial derivative, indicated by éz/Ox, represents the rate 
of change in the variable z with respect to (a unit increase in) the variable x. We 
shall also utilize the symbol 0z/0x to indicate the vector of partial derivatives 
of z with respect to each element of the vector x. That is, if x = (x1,%,...,%,), 
then 
Oz _ (| & Oz Oz 
ox x,” Ox.” Ax, J 
Also, we shall sometimes consider the partial derivative of one vector with 
respect to another vector, such as Oy/Ox. If y = (4,¥2.+5¥_) and x = 
(X1,X2,...5%,), then 


oy OY oy, 
Ox, OX Ox, 
by Cyr Vn Oy2 
ex = Ox, oe ox, n 
Ym Ym Ym 
ox 1 Ox 2 Ox n 
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Note that if z is a function of the vector x = (x1,X>,...,X,), then 0z/0x is 


called the gradient of z. 

We shall, when necessary, use (a, 5) to refer to the open interval a <x < 
b, and [a, ] to refer to the closed interval a < x < b. Finally we shall utilize 
the standard set operators U, ©, c, and €¢ to refer to union, intersection, set 
inclusion, and set membership, respectively. 


EXERCISES 


[1.1] Fred has $5000 to invest over the next five years. At the beginning of each 
year he can invest money in one— or two-year time deposits. The bank pays 4 
percent interest on one-year time deposits and 9 percent (total) on two-year 
time deposits. In addition, West World Limited will offer three-year certificates 
starting at the beginning of the second year. These certificates will return 15 
percent (total). If Fred reinvests his money that is available every year, 
formulate a linear program to show him how to maximize his total cash on hand 
at the end of the fifth year. 


[1.2] A manufacturer of plastics is planning to blend a new product from four 
chemical compounds. These compounds are mainly composed of three 
elements: A, B, and C. The composition and unit cost of these chemicals are 
shown in the following table: 


CHEMICAL COMPOUND 1 2 3 4 
Percentage A 35 15 35 25 
Percentage B 20 65 35 40 
Percentage C 40 15 25 30 
Cost/kilogram 20 30 20 15 


The new product consists of 25 percent element A, at least 35 percent element 
B, and at least 20 percent element C. Owing to side effects of compounds | and 
2, they must not exceed 25 percent and 30 percent, respectively, of the content 
of the new product. Formulate the problem of finding the least costly way of 
blending as a linear program. 


[1.3] An agricultural mill manufactures feed for cattle, sheep, and chickens. This 
is done by mixing the following main ingredients: corn, limestone, soybeans, 
and fish meal. These ingredients contain the following nutrients: vitamins, 
protein, calcium, and crude fat. The contents of the nutrients in standard units 
for each kilogram of the ingredients are summarized in the following table: 


NUTRIENT UNITS 
INGREDIENT VITAMINS PROTEIN CALCIUM CRUDE FAT 
Com 8 10 6 8 
Limestone 6 5 10 6 
Soybeans 10 12 6 6 
Fish meal 4 8 6 9 


The mill is contracted to produce 12, 8, and 9 (metric) tons of cattle feed, sheep 
feed, and chicken feed. Because of shortages, a limited amount of the 
ingredients is available—namely, 9 tons of corn, 12 tons of limestone, 5 tons of 
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soybeans, and 6 tons of fish meal. The price per kilogram of these ingredients is, 
respectively, $0.20, $0.12, $0.24, and $0.12. The minimal and maximal units of 
the various nutrients that are permitted is summarized below for a kilogram of 
the cattle feed, the sheep feed, and the chicken feed. 


NUTRIENT UNITS 
VITAMINS PROTEIN CALCIUM CRUDE FAT 
PRODUCT MIN MAX MIN MAX MIN MAX MIN MAX 
Cattle feed 6 20 6 00 7 00 4 8 
Sheep feed 6 i) 6 i) 6 00 4 6 
Chicken feed 4 6 6 20 6 00 4 5 


Formulate this feed—mix problem so that the total cost is minimized. 


[1.4] Consider the problem of locating a new machine to an existing layout 
consisting of four machines. These machines are located at the following 


coordinates in two-dimensional space: i) Ey ei and ia Let the 


coordinates of the new machine be (2), Formulate the problem of finding an 


optimal location as a linear program for each of the following cases: 


a. The sum of the distances from the new machine to the four machines 
is minimized. Use the street distance (also known as Manhattan 
distance or rectilinear distance); for example, the distance from 
(2) to the first machine located at (?) is |x, - 3] + [xy - J]. 

b. Because of various amounts of flow between the new machine and 
the existing machines, reformulate the problem where the sum of the 
weighted distances is minimized, where the weights corresponding 
to the four machines are 6, 4, 7, and 2, respectively. 

c. In order to avoid congestion, suppose that the new machine must be 
located in the square {(x),x.):-1 < x, $ 2, O< x) < J}. Form— 
ulate Parts (a) and (b) with this added restriction. 

d. Suppose that the new machine must be located so that its distance 
from the first machine does not exceed 2. Formulate the problem 
with this added restriction. 


[1.5] The technical staff of a hospital wishes to develop a computerized menu— 
planning system. To start with, a lunch menu is sought. The menu is divided 
into three major categories: vegetables, meat, and dessert. At least one equi- 
valent serving of each category is desired. The cost per serving of some 
suggested items as well as their content of carbohydrates, vitamins, protein, and 
fats is summarized below: 
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CARBO- COST IN 
HYDRATES VITAMINS PROTEIN’ FATS $/SERVING 
Vegetables 
Peas 1 3 1 0 0.10 
Green beans 1 5 2 0 0.12 
Okra 1 5 1 0 0.13 
Corn 2 6 | 2 0.09 
Macaroni 4 2 1 1 0.10 
Rice 5 1 1 1 0.07 
Meat 
Chicken 2 1 3 1 0.70 
Beef 3 8 5 2 1.20 
Fish 3 6 6 | 0.63 
Dessert 
Orange 1 3 1 0 0.28 
Apple 1 2 0 0 0.42 
Pudding 1 0 0 0 0.15 
Jello 1 0 0 0 0.12 


Suppose that the minimal requirements of carbohydrates, vitamins, protein, and 
fats per meal are respectively 5, 10, 10, and 2. 


a. Formulate the menu-planning problem as a linear program. 


b. Many practical aspects have been left out in the foregoing model. 
These include planning the breakfast, lunch, and supper menus 
together, weekly planning so that different varieties of food are used, 
and special menus for patients on particular diets. Discuss in detail how 
these aspects can be incorporated in a comprehensive menu—planning 
system. 


[1.6] A cheese firm produces two types of cheese: swiss cheese and sharp 
cheese. The firm has 60 experienced workers and would like to increase its 
working force to 90 workers during the next eight weeks. Each experienced 
worker can train three new employees in a period of two weeks during which 
the workers involved virtually produce nothing. It takes one man-hour to 
produce 10 pounds of Swiss cheese and one man—hour to produce 6 pounds of 
sharp cheese. A work week is 40 hours. The weekly demands (in 1000 pounds) 
are summarized below: 


WEEK 
CHEESE TYPE 1 2 3 4 5 6 7 8 
Swiss cheese 11 12 #13 «18 «14 «18 «20 =6« 20 
Sharp cheese 8 8 10 8 12 #13 #12 «#12 


Suppose that a trainee receives the same full salary as an experienced worker. 
Further suppose that overaging destroys the flavor of the cheese, so that 
inventory is limited to one week. How should the company hire and train its 
new employees so that the labor cost is minimized over this 8-week period? 
Formulate the problem as a linear program. 
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[1.7] A company wishes to plan its production of two items with seasonal 
demands over a 12—month period. The monthly demand of item | is 100,000 
units during the months of October, November, and December; 10,000 units 
during the months of January, February, March, and April; and 30,000 units 
during the remaining months. The demand of item 2 is 50,000 during the 
months of October through February and 15,000 during the remaining months. 
Suppose that the unit product cost of items 1 and 2 is $5.00 and $8.50, 
respectively, provided that these were manufactured prior to June. After June, 
the unit costs are reduced to $4.50 and $7.00 because of the installation of an 
improved manufacturing system. The total units of items 1 and 2 that can be 
manufactured during any particular month cannot exceed 120,000 for Jan—Sept, 
and 150,000 for Oct-Dec. Furthermore, each unit of item 1 occupies 2 cubic feet 
and each unit of item 2 occupies 4 cubic feet of inventory space. Suppose that 
the maximum inventory space allocated to these items is 150,000 cubic feet and 
that the holding cost per cubic foot during any month is $0.20. Formulate the 
production scheduling problem so that the total cost of production and inventory 
is minimized. 

[1.8] A textile mill produces five types of fabrics. The demand (in thousand 
yards) over a quarter—year time horizon for these fabrics is 16, 48, 37, 21, and 
82, respectively. These five fabrics are woven, finished, and sold in the market 
at prices 0.9, 0.8, 0.8, 1.2, and 0.6 $/per yard, respectively. Besides weaving and 
finishing the fabrics at the mill itself, the fabrics are also purchased woven from 
outside sources and are then finished at the mill before being sold. If the 
unfinished fabrics are purchased outside, the costs in $/per yard for the five 
fabrics are 0.8, 0.7, 0.75, 0.9, and 0.7, respectively. If produced at the mill itself, 
the respective costs are 0.6, 0.5, 0.6, 0.7, and 0.3 $/per yard. There are two types 
of looms that can produce the fabrics at the mill, that is, there are 10 Dobbie 
looms and 80 regular looms. The production rate of each Dobbie loom is 4.6, 
4.6, 5.2, 3.8, and 4.2 yards per hour for the five fabrics. The regular looms have 
the same production rates as the Dobbie looms, but they can only produce fabric 
types 3, 4, and 5. Assuming that the mill operates seven days a week and 24 
hours a day, formulate the problem of optimally planning to meet the demand 
over a quarter—year horizon as a linear program. Is your formulation a 
transportation problem? If not, reformulate the problem as a transportation 
problem. 


[1.9] A steel manufacturer produces four sizes of I beams: small, medium, large, 
and extra large. These beams can be produced on any one of three machine 
types: A, B, and C. The lengths in feet of the I beams that can be produced on 
the machines per hour are summarized below: 


MACHINE 
BEAM A B C 
Small 350 650 850 
Medium 250 400 700 
Large 200 350 600 
Extra large 125 200 325 


Assume that each machine can be used up to 50 hours per week and that the 
hourly operating costs of these machines are respectively $30.00, $50.00, and 
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$80.00. Further suppose that 12,000, 6000, 5000, and 7000 feet of the different 
size I beams are required weekly. Formulate the machine scheduling problem as 
a linear program. 


[1.10] An oil refinery can buy two types of oil: light crude oil and heavy crude 
oil. The cost per barrel of these types is respectively $20 and $15. The following 
quantities of gasoline, kerosene, and jet fuel are produced per barrel of each type 
of oil. 


GASOLINE KEROSENE JET FUEL 


Light crude oil 0.4 0.2 0.35 
Heavy crude oil 0.32 0.4 0.2 


Note that 5 percent and 8 percent, respectively, of the light and heavy crude oil 
are lost during the refining process. The refinery has contracted to deliver | 
million barrels of gasoline, 500,000 barrels of kerosene, and 300,000 barrels of 
jet fuel. Formulate the problem of finding the number of barrels of each crude 
oil that satisfies the demand and minimizes the total cost as a linear program. 


[1.11] A lathe is used to reduce the diameter of a steel shaft whose length is 36 in. 
from 14 in. to 12 in. The speed x, (in revolutions per minute), the depth feed x, 
(in inches per minute), and the length feed x3 (in inches per minute) must be 
determined. The duration of the cut is given by 36/x.x3. The compression and 
side stresses exerted on the cutting tool are given by 30x, + 4500 x. and 40x, + 
5000 x. + 5000x3; pounds per square inch, respectively. The temperature (in 
degrees Fahrenheit) at the tip of the cutting tool is 200 + 0.5 x, + 150( x) + x3). 
The maximum compression stress, side stress, and temperature allowed are 
150,000 psi, 100,000 psi, and 800°F, respectively. It is desired to determine the 
speed (which must be in the range from 600 rpm to 800 rpm), the depth feed, and 
the length feed such that the duration of the cut is minimized. In order to use a 
linear model, the following approximation is made. Since 36/x,x3 1s minimized if 


and only if x2x3 is maximized, it was decided to replace the objective by the 
maximization of the minimum of x, and x3. Formulate the problem as a linear 


model and comment on the validity of the approximation used in the objective 
function. 


[1.12] A television set manufacturing firm has to decide on the mix of color and 
black-and-white TVs to be produced. A market research indicates that, at most, 
2000 units and 4000 units of color and black-and-white TVs can be sold per 
month. The maximum number of man-hours available is 60,000 per month. A 
color TV requires 20 man-hours and a black-and-white TV requires 15 man— 
hours to manufacture. The unit profits of the color and black-and-white TVs are 
$60 and $30, respectively. It is desired to find the number of units of each TV 
type that the firm must produce in order to maximize its profit. Formulate the 
problem as a linear program. 


[1.13] A production manager is planning the scheduling of three products on 


four machines. Each product can be manufactured on each of the machines. The 
unit production costs (in $) are summarized below. 
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MACHINE 


PRODUCT 


The time (in hours) required to produce a unit of each product on each of the 
machines is summarized below. 
MACHINE 
PRODUCT 


Suppose that 3000, 6000, and 4000 units of the products are required, and that 
the available machine-hours are 1500, 1200, 1500, and 2000, respectively. 
Formulate the scheduling problem as a linear program. 


[1.14] A furniture manufacturer has three plants that need 500, 700, and 600 
tons of lumber weekly. The manufacturer may purchase the lumber from three 
lumber companies. The first two lumber manufacturers virtually have an 
unlimited supply and, because of other commitments, the third manufacturer 
cannot ship more than 500 tons weekly. The first lumber manufacturer uses rail 
for transportation and there is no limit on the tonnage that can be shipped to the 
furniture facilities. On the other hand, the last two lumber companies use trucks 
that limit the maximum tonnage that can be shipped to any of the furniture 
companies to 200 tons. The following table gives the transportation cost from 
the lumber companies to the furniture manufacturers ($ per ton). 


LUMBER FURNITURE FACILITY 
COMPANY 


Formulate the problem as a linear program. 


[1.15] A company manufactures an assembly consisting of a frame, a shaft, and 
a ball bearing. The company manufactures the shafts and frames but purchases 
the ball bearings from a ball bearing manufacturer. Each shaft must be 
processed on a forging machine, a lathe, and a grinder. These operations require 
0.6 hour, 0.3 hour, and 0.4 hour per shaft, respectively. Each frame requires 0.8 
hour on a forging machine, 0.2 hour on a drilling machine, 0.3 hour on a milling 
machine, and 0.6 hour on a grinder. The company has 5 lathes, 10 grinders, 20 
forging machines, 3 drillers, and 6 millers. Assume that each machine operates a 
maximum of 4500 hours per year. Formulate the problem of finding the 
maximum number of assembled components that can be produced as a linear 
program. 


[1.16] A corporation has $30 million available for the coming year to allocate to 
its three subsidiaries. Because of commitments to stability of personnel em- 
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ployment and for other reasons, the corporation has established a minimal level 
of funding for each subsidiary. These funding levels are $3 million, $5 million, 
and $8 million, respectively. Owing to the nature of its operation, subsidiary 2 
cannot utilize more than $17 million without major new capital expansion. The 
corporation is unwilling to undertake such an expansion at this time. Each 
subsidiary has the opportunity to conduct various projects with the funds it 
receives. A rate of return (as a percent of investment) has been established for 
each project. In addition, certain projects permit only limited investment. The 
data of each project are given below: 


RATE OF UPPER LIMIT OF 
SUBSIDIARY PROJECT RETURN INVESTMENT 
1 T% $6 million 
l 2 5% $5 million 
3 8% $9 million 
4 5% $7 million 
2 5 T% $10 million 
6 9% $4 million 
3 7 10% $6 million 
8 8% $3 million 


Formulate this problem as a linear program. 


[1.17] A 10—acre slum in New York City is to be cleared. The officials of the 
city must decide on the redevelopment plan. Two housing plans are to be 
considered: low-income housing and middle-income housing. These types of 
housing can be developed at 20 and 15 units per acre, respectively. The unit 
costs of the low— and middle-income housing are $17,000 and $25,000. The 
lower and upper limits set by the officials on the number of low-income 
housing units are 80 and 120. Similarly, the number of middle-income housing 
units must lie between 40 and 90. The combined maximum housing market 
potential is estimated to be 190 (which is less than the sum of the individual 
market limits due to the overlap between the two markets). The total mortgage 
committed to the renewal plan is not to exceed $2.5 million. Finally, it was 
suggested by the architectural adviser that the number of low-income housing 
units be at least 50 units greater than one-half the number of the middle-income 
housing units. 


a. Formulate the minimum cost renewal planning problem as a linear 
program and solve it graphically. 

b. Resolve the problem if the objective is to maximize the number of 
houses to be constructed. 


[1.18] Consider the following problem of launching a rocket to a fixed altitude b 
in a given time T while expending a minimum amount of fuel. Let u(#) be the 
acceleration force exerted at time ¢ and let y(t) be the rocket altitude at time 4 
The problem can be formulated as follows: 


Minimize ie |u(e)| dt 
subjectto j(t)=u(t)-g 
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VT) = 5 
y(t) > 0, ¢ € [0,7], 


where g is the gravitational force and } is the second derivative of the altitude 


y. Discretize the problem and reformulate it as a linear programming problem. 
In particular, formulate the problem for T = 15, b = 20, and g = 32. (Hint: 
Replace the integration by a proper summation and the differentiation by 


difference equations. Make the change of variables |x ,| = x,, Vj, based on the 
discretization, and note that x peu and x; 2 -u;.) 


[1.19] Consider the Multiperiod Coal Blending and Distribution Problem 
presented in Section |.2. For the developed model, we have ignored the effect 
of any production and distribution decisions made prior to period ¢ = 1 that 
might affect the present horizon problem, and we have also neglected to 
consider how production decisions during the present horizon might impact 
demand beyond period 7 (especially considering the shipment lag phenomenon). 
Provide a detailed discussion on the impact of such considerations, and suggest 
appropriate modifications to the resulting model. 


[1.20] Consider the Multiperiod Coal Blending and Distribution Problem 
presented in Section 1.2, and assume that the shipment lag is zero (but that the 
initial storage at the silos is still assumed to be dissipated in three periods). 


Defining the principal decision variables as yf, = amount (tons) of coal 
shipped from mine i through silo 7 to customer & during a particular time period 
t, and Yoke = amount (tons) of coal that is in initial storage at silo j, which is 


shipped to customer k in period t (where 7 = 1, 2, 3), and defining all other 
auxiliary variables as before, reformulate this problem as a linear program. 
Discuss its size, structure, and assumptions relative to the model formulated in 
Section 1.2. 


[1.21] Consider the Multiperiod Coal Blending and Distribution Problem pre— 
sented in Section 1.2. Suppose that we define the principal decision variables as 


Vit = amount (tons) of coal shipped from mine i to silo / in period 4 one = 


amount (tons) of coal shipped from silo / to customer & in period ¢; and yout = 


amount (tons) of coal that is in initial storage at silo 7, which is shipped to 
customer & in period ¢ (where ¢ = 1, 2, 3), with all other auxiliary decision 
variables being defined as before. Assuming that coal shipped from any mine i 
to a silo j will essentially affect the ash and sulfur content at silo 7 for only three 
periods as before (defined as the shipment lag), derive expressions for the 
percentage ash and percentage sulfur contents of the coal that would be 
available for shipment at silo 7 during each period ¢, in terms of the foregoing 
decision variables. Discuss how you might derive some fixed constant estimates 
for these values. Using these estimated constant values, formulate the coal 
distribution and blending problem as a linear program in terms of the just— 
mentioned decision variables. Discuss the size, structure, and assumptions of 
this model relative to the model formulated in Section 1.2. Also, comment on 
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the nature of the resulting model if we used the actual expressions for the ash 
and sulfur content of coal at each silo 7 during each period ¢, in lieu of the 
constant estimates. 

[1.22] A region is divided into m residential and central business districts. Each 
district is represented by a node, and the nodes are interconnected by links 
representing major routes. People living in the various districts go to their 
business in the same and/or at other districts so that each node attracts and/or 


generates a number of trips. In particular, let a be the number of trips 


generated at node i with final destination at node / and let b; be the time to 


travel from node i to node /. It is desired to determine the routes to be taken by 
the people living in the region. 


a. Illustrate the problem by a suitable network. 
b. Develop some measures of effectiveness for this traffic assignment 
problem, and for each measure, devise a suitable model. 
[1.23] Consider the problem of scheduling court hearings over a planning 
horizon consisting of n periods. Let b ); be the available judge—hours in period /, 


hy; be the number of hearings of class i arriving in period j, and a; be the 


number of judge—hours required to process a hearing of class i. It is desired to 
determine the number of hearings x,; of class i processed in period /. 


a. Formulate the problem as a linear program. 
b. Modify the model in Part (a) so that hearings would not be delayed 
for too long. 

[1.24] Consider the following multiperiod, multiproduct Production—Inventory 
Problem. Suppose that we are examining T periods ¢ = 1,..., T, and some n 
products i = 1,..., 7. There is an initial inventory of y,9 that is available at hand 
for each product i, i = 1,..., 2. We need to principally determine the level of 
production for each product i = 1,..., 7 during each of the periods ¢ = 1,..., 7, so 
as to meet the forecasted demand d;, for each product i during each period ¢ 
when it occurs, at a minimal total cost. To take advantage of varying demands 
and costs, we are permitted to produce excess quantities to be stored for later 
use. However, each unit of product i consumes a storage space s;, and incurs a 
storage cost of c;, during period ¢, for i = 1,..., n, = 1,..., T. The total storage 
space anticipated to be available for these m products during period ¢ is S,, f= 1 
,-., 7. Furthermore, each unit of product i requires 4; hours of labor to produce, 
where the labor cost per hour when producing a unit of product i during period ¢ 
is given by @,,, for i = l,..., n, t = 1,..., 7. The total number of labor hours 
available to manufacture these n products during period ¢ is given by H,. 


Formulate a linear program to solve this production—inventory control system 
problem, prescribing production as well as inventory levels for each product 
over each time period. 
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[1.25] Suppose that there are m sources that generate waste and n disposal sites. 
The amount of waste generated at source i is a; and the capacity of site j is b;. 
It is desired to select appropriate transfer facilities from among K candidate 
facilities. Potential transfer facility k has a fixed cost f;, capacity g,, and unit 
processing cost a per ton of waste. Let cx and Cc; be the unit shipping costs 
from source i to transfer station k and from transfer station k to disposal site /, 
respectively. The problem is to choose the transfer facilities and the shipping 
pattern that minimize the total capital and operating costs of the transfer stations 
plus the transportation costs. Formulate this distribution problem. (Hint: Let y, 
be | if transfer station k is selected and 0 otherwise.) 


[1.26] A governmental planning agency wishes to determine the purchasing 
sources for fuel for use by m depots from among m bidders. Suppose that the 
maximum quantity offered by bidder i is a; gallons and that the demand of depot j 


is b; gallons. Let c; be the unit delivery cost for bidder i to the jth depot. 


a. Formulate the problem of minimizing the total purchasing cost as a 
linear program. 

b. Suppose that a discount in the unit delivery cost is offered by bidder i 
if the ordered quantity exceeds the level a@;. How would you 


incorporate this modification in the model developed in Part (a)? 


[1.27] The quality of air in an industrial region largely depends on the effluent 
emission from n plants. Each plant can use m different types of fuel. Suppose 
that the total energy needed at plant j is 5; British Thermal Units per day and 


that cj; is the effluent emission per ton of fuel type i at plant j. Further suppose 


that fuel type i costs c; dollars per ton and that each ton of this fuel type generates 
a; British Thermal Units at plant j. The level of air pollution in the region is not 
to exceed 5 micrograms per cubic meter. Finally, let 7 j be a meteorological 
parameter relating emissions at plant/ to air quality in the region. 


a. Formulate the problem of determining the mix of fuels to be used at 
each plant. 

b. How would you incorporate technological constraints that prohibit 
the use of certain mixes of fuel at certain plants? 

c. How could you ensure equity among the plants? 


[1.28] For some industry, let p(Q), Q = 0, be an inverse demand curve, that is, 


P(Q) is the price at which a quantity Q will be demanded. Let {Q), Q = 0, bea 


supply or marginal cost curve, that is, {Q) is the price at which the industry is 
willing to supply Q units of the product. Consider the problem of determining a 


(perfect competition) equilibrium price and production quantity p* and Q” for 
p() and f(-) of the type shown in Figure 1.14 via the intersection of these 
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supply and demand curves. Note that Q* is obtainable by finding that Q, which 


maximizes the area under the demand curve minus the area under the supply 
curve. Now, suppose that /(-) is not available directly, but being a marginal 
cost curve, is given implicitly by 

IP f(x) de = min{cy: Ay=b, ay =Q,y > 0}, 
where A is m xn, @ andc are | x n, and y is an n-vector representing a set of 
production activities. Further, suppose that the demand curve p(-) is approx— 
imated using the steps shown in the figure. (Let there be s steps used, each of 


height H, and width W, for i = 1,..., s.) With this approximation, formulate the 
problem of determining the price-quantity equilibrium as a linear program. 


RQ) 


Q 


oO" 


Figure 1. 14. Supply demand equilibrium for Exercise 1.28 


[1.29] Consider the following two-stage stochastic program with recourse. 
Suppose that in a time-stage process, we need to make some immediate decision 
(“here and now”) that is represented by the decision variable vector x > 0. (For 
example, x might represent some investment or production decisions.) This 
decision must satisfy the constraints Ax = b. Subsequent to this, a random demand 
vector d, say, will be realized according to some probability distribution. Based on 
the initial decision x and the realization d, we will need to make a recourse 
decision y = 0 (e.g., some subsequent additional production decisions), such that 
the constraint Dx + Ey = d is satisfied. The total objective cost for this pair of 
decisions is given by ¢;x + czy. Assume that the random demand vector d can 


take on a discrete set of possible values d,, dp, ..., d, with respective known 


q 
probabilities p,, P25+sPq> where >) p, =1, p, > 0, Vr =1,....g¢. (Accord— 
#=1 


ingly, note that each realization d, will prompt a corresponding recourse decision 
y,-) Formulate the problem of minimizing the total expected cost subject to the 


initial and recourse decision constraints as a linear programming problem. Do you 
observe any particular structure in the overall constraint coefficient matrix with 
respect to the possible nonzero elements? (Chapter 7 explores ways for solving 
such problems.) 


[1.30] Consider the following linear programming problem. 
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Minimize x, — 2x, — 3x3 
subjectto -x, + 3x, + x < 13 
xy + 2x. + 3x3 > 12 
2x4 -— X + xX = 4 
Xs x9 unrestricted 
x3 <-3. 


a. Reformulate the problem so that it is in standard format. 
b. Reformulate the problem so that it is in canonical format. 
c. Convert the problem into a maximization problem. 


{1.31] Consider the following problem: 


Maximize My - xX 
subjectto -x, + 3x, < 0 
—3x, + 2x, 2 -3 
xX, x2 2 0. 


a. Sketch the feasible region in the (x, x2) space. 
b. Identify the regions in the (x, x.) space where the slack variables 
x3 and x4, say, are equal to zero. 


c. Solve the problem geometrically. 
d. Draw the requirement space and interpret feasibility. 


[1.32] Consider the feasible region sketched in Figure 1.55. Geometrically, 
identify conditions on the objective gradient vector ¢ for which the different 
points in the feasible region will be optimal and identify the vectors ¢ for which 
no optimum exists. (Assume a minimization problem.) 


[1.33] Sketch the feasible region of the set {x: Ax < b} where A and b are as 
given below. In each case, state whether the feasible region is empty or not and 
whether it is bounded or not. 


[1.34] Consider the following problem: 
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a. 
b. 


Maximize 3x, + x, 
subject to -x, + 2x) 


0 
x2 4. 


IA IA 


Sketch the feasible region. 
Verify that the problem has an unbounded optimal solution value. 


[1.35] Consider the following problem: 


a. 
b. 
c. 


Maximize 2x, + 3x, 
subjectto x4 + X 
4x, + 6x, 
X> X42 


VIA IA 
COON 


Sketch the feasible region. 
Find two alternative optimal extreme (corner) points. 
Find an infinite class of optimal solutions. 


[1.36] Consider the following problem: 


ss 


Maximize -x, -— X27 + 2x3, + X4 

subjectto 2x, + XxX + %3 + X%4 2 6 
X + 2x - 2x3 + X% S 4 
X> x2, X3, X4 > 0. 


Introduce slack variables and draw the requirement space. 

Interpret feasibility in the requirement space. 

You are told that an optimal solution can be obtained by having at 
most two positive variables while all other variables are set at zero. 
Utilize this statement and the requirement space to find an optimal 
solution. 


[1.37] Consider the following problem: 


Maximize 3x, + 6X 

subjectto -x, + 2x, < 2 
2x, + x2 < 0 
x + 2X2 < 4 
xX}; x2 > 0. 


Graphically identify the set of all alternative optimal solutions to this 
problem. 

Suppose that a secondary priority objective function seeks to 
maximize -3x,; + x. over the set of alternative optimal solutions 
identified in Part (a). What is the resulting solution obtained? 


What is the solution obtained if the priorities of the foregoing two 
objective functions is reversed? 


[1.38] Consider the problem: Minimize ex subject to Ax > b, x > 0. Suppose that 
one component of the vector b, say b;, is increased by one unit to 5; + 1. 


a. 
b. 


What happens to the feasible region? 
What happens to the optimal objective value? 


[1.39] From the results of the previous problem, assuming éz*/ 0b; exists, is it < 
0, = 0, or > 0? 
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[1.40] Solve Exercises 1.38 and 1.39 if the restrictions Ax > b are replaced by 
Ax <b. 


[1.41] Consider the problem: Minimize ex subject to Ax > b, x > 0. Suppose that 
a new constraint is added to the problem. 

a. What happens to the feasible region? 

b. What happens to the optimal objective value z”? 


[1.42] Consider the problem: Minimize ex subject to Ax => b, x > 0. Suppose that 
a new variable is added to the problem. 

a. What happens to the feasible region? 

b. What happens to the optimal objective value z”? 


[1.43] Consider the problem: Minimize ex subject to Ax > b, x > 0. Suppose that 
a constraint, say constraint i, is deleted from the problem. 

a. What happens to the feasible region? 

b. What happens to the optimal objective value z*? 


[1.44] Consider the problem: Minimize ex subject to Ax > b, x > 0. Suppose that 
a variable, say, x;,, is deleted from the problem. 


a. What happens to the feasible region? 
b. What happens to the optimal objective value z”? 


NOTES AND REFERENCES 


1. Linear programming and the simplex method were developed by Dantzig 
in 1947 in connection with military planning. A great deal of work has 
influenced the development of linear programming, including World War 
II operations and the need for scheduling supply and maintenance 
operations as well as training of Air Force personnel: Leontief’s input— 
output model [1951], von Neumann’s Equilibrium Model [1937], 
Koopmans’ Model of Transportation [1949], the Hitchcock transportation 
problem [1941], the work of Kantorovich [1958], von Neumann-— 
Morgenstern game theory [1944], and the rapid progress in electronic 
computing machines have also impacted the field of linear programming. 
The papers by Dantzig [1982] and by Albers and Reid [1986] provide 
good historical developments. 


2: Linear programming has found numerous applications in the military, the 
government, industry, and urban engineering. See Swanson [1980], for 
example. 

3. Linear programming is also frequently used as a part of general 


computational schemes for solving nonlinear programming problems, 
discrete programs, combinatorial problems, problems of optimal control, 
and programming under uncertainty. 

4. Exercise 1.8 is derived from Camm et al. [1987], who discuss some 
modeling issues. For more detailed discussions, see Woolsey and 
Swanson [1975], Woolsey [2003], and Swanson [1980]. The Multiperiod 
Coal Blending and Distribution Problem discussed in Section 1.2, and 
Exercises 1.19-1.21, are adapted from Sherali and Puri [1993] (see also 
Sherali and Saifee [1993]). For a discussion on structured modeling and 
on computer—assisted/artificial intelligence approaches to linear 
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programming modeling, see Geoffrion [1987], Greenberg [1983], and 
Murphy and Stohr [1986], for example. 

5. For discussions on diagnosing infeasibilities and related /rreducible 
Infeasible System (IIS) issues, see Amaldi et al. [2003], Greenberg 
[1996], Greenberg and Murphy [1991], and Parker and Ryan [1996]. 

6. For an excellent discussion on modeling issues and insights related to 
formulating and solving real-world problems, the reader is referred to 
Brown and Rosenthal [2008] (also, see the other references cited therein), 
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TWO: LINEAR ALGEBRA, CONVEX 
ANALYSIS, AND POLYHEDRAL SETS 


In this chapter we review some basic results from linear algebra and convex 
analysis. These results will be used throughout the book. The reader may skip 
any sections of this chapter, according to his or her familiarity with the subject 
material. Sections 2.1 and 2.2 review some elementary results from vector and 
matrix algebra. In Section 2.3 we discuss the solvability of a system of linear 
equations and introduce an important notion of basic solutions. The remaining 
sections discuss results from convex analysis, including the notions of convex 
sets, convex and concave functions, convex cones, hyperplanes, and polyhedral 
sets. The sections on polyhedral sets, which are sets over which linear programs 
are solved, and their representation in terms of extreme points and extreme 
directions are very important in linear programming, and hence they deserve 
thorough study. In particular, Section 2.6 provides geometric insights into the 
structure of polyhedral sets and develops its various characterizations using only 
fundamental definitions. 


2.1 VECTORS 


An n-vector is a row or a column array of m numbers. For example, a = (1, 2, 3, 
—1, 4) is a row vector of size n = 5, and a= (_)) is a column vector of size n = 


2. Row and column vectors are denoted by lowercase boldface letters, such as 
a, b, c. Whether a vector is a row or a column vector will be clear from the 
context. Figure 2.1 shows some vectors in a two-dimensional space. Each 
vector can be represented by a point or by a line from the origin to the point, 
with an arrowhead at the end point of the line. 


Figure 2.1. Some examples of vectors. 
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Special Vectors 
ZERO VECTOR 


The zero vector, denoted by 0, is a vector with all components equal to zero. 
This vector is also referred to as the origin. 


ith UNIT VECTOR 


This is a vector having zero components, except for a 1 in the ith position. This 
vector is denoted by e; and is sometimes called the ith coordinate vector. 


ith position 


L 
e; = (0,0....,1,...,0,0). 


SUM VECTOR 


This is a vector having each component equal to one. The sum vector is denoted 
by 1. (Sometimes, the vector e is used to denote this vector.) 


Addition and Multiplication of Vectors 


ADDITION 


Two vectors of the same size can be added, where addition is performed 
componentwise. To illustrate, let a; and a, be the following two n-vectors: 


a, = (a 1242] 2-9 Ay ) 


a= (42,422 5++-5 Ay?) 
Then the addition of a; and az, denoted by a; + a, is the following vector: 
ay T a= (a1 + A272, AQ, + A77,..-, Ay] + An2): 


The operation of vector addition is illustrated in Figure 2.2. Note that a; + a) is 
the diagonal of the parallelogram with sides a, and ap. It is obtained by step— 


ping along a, and then ap, or vice versa. 


SCALAR MULTIPLICATION 
The operation of multiplying a vector a with a scalar k is performed component-— 
wise. If a = (a@,a),...,a,), then the vector ka = (ka,,kap,...,ka,,). This opera— 


tion is shown in Figure 2.3. If k > 0, then ka points in the same direction as a. 
On the other hand, if k < 0, then ka points in the opposite direction. 
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Figure 2.2. Vector addition. 


Inner Product 


Any two n-vectors a and b can be multiplied. The result of this multiplication is 
a real number called the inner or dot product of the two vectors. It is defined as 
follows: 


n 
ab = ab, + azby spoxes se Andy = py a;b;. 
j=l 


For example, if a = (1, -1) and b = (4), then ab = -3. 


Norm of a Vector 


Various norms (measures of size) of a vector can be used. We shall use here the 
Euclidean norm. This norm is the square root of the inner product of the vector 


and itself. In other words, the norm of a vector a, denoted by |al|, is given by 


27214; - Note that 


Ja + bf? = [al + [bl + 2ab 
for any two vectors a and b of the same size. 


Schwartz Inequality 


Given two vectors a and b of the same size, the following inequality, called the 
Schwartz inequality, holds: 


[ab] < |al||b]. 
To illustrate, let a = (0, 2) and b = (2). Then ab = 8, whereas |al| = 2 and |[b|| = 


5. Clearly 8 < 2 x 5. In fact, given nonzero vectors a and b, the ratio of the 
inner product ab to [al| |[b|| provides a measure of the angle @ between the two 
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ka,k>0O (here, k> 1) 


Figure 2.3. Scalar multiplication. 


vectors. In particular, cos@ = ab/|lal| bl. Of course, if ab = 0, then cos@ = 0; 


that is, the two vectors a and b are orthogonal or normal or perpendicular to 
each other. Figure 2.4 shows two orthogonal vectors a and b. 


Euclidean Space 


An n-dimensional (real) Euclidean space, denoted by R”, is the collection of 


all vectors of dimension n. Addition and scalar multiplication of vectors in R” 
were defined previously. Also, associated with any vector in R” is its norm, and 
associated with any two vectors in R” is their inner product, defined previously. 


Linear and Affine Combinations 

A vector b in R” is said to be a linear combination of a,,43,...,a, in R”, ifb 
= rk Ajay, where 4, Ay,...,4, are real numbers. If, in addition, Lay = 
1, then b is said to be an affine combination of a,,a9,...,a,. 

Linear and Affine Subspaces 

A linear subspace S; of R” is a subset of R” such that if a; and az € S;, 
then every linear combination of a; and ay belongs to S;. Similarly, an affine 
subspace S4 of R” is asubset of R” such that if a; and ay € S4, then every 
affine combination of a, and a, belongs to S4. 

Linear Independence 


A collection of vectors a),a,...,.a; of dimension 7 is called linearly inde- 
pendent if 
Ajaj = 0 implies that Aj = 0, for all j = 1.,..., k. 


1 J 


TM=> 
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a=(-1,3) 


b=(6,2) 


Figure 2.4. Orthogonal vectors. 


For example, let a; = (1, 2) and ay = (—I,1). These two vectors are linearly 
independent because 4, (1,2) + 2,(-1, 1) = (0,0) implies that 4, = 2, =0.A 
collection of vectors is called linearly dependent if they are not linearly 
independent. Therefore, aj,a,...,a, are linearly dependent if there exist 


Ay Ags.n-5 Ay, not all zero, such that D_,Aja; = 0. For example, let a, = (1, 2, 
3), a, =(-1,1,-1), and a3 = (0,3,2). These three vectors are linearly 
dependent because Aa, + Ayan + A,a3 =O for A, = A, = 1 and A, = -1. 
Spanning Set 

A collection of vectors a,,@,...,a, in R” is said to span R” if any vector in 
R" can be represented as a linear combination of @),€>,...,.a;. In other words, 


given any vector b in R”, we must be able to find scalars Ay, Ags.+5 Ay such that 


b= yi 


To illustrate, let n = 2, and consider a, = (1,0), ay = (-1,3), and a3 = 


14;4;- 


(2, 1). The vectors a,,a7,a3 span R’, since any vector b in R can be repre— 


sented as a linear combination of these vectors. For example, b = (5, 5)) can 
1 1 

be represented as A;a; + Aan + A;a3 where A, = b + 32 An = 302 and 
Az = 0. In this case, the representation is not unique. Another representation is 
given by letting 4, = b, — 2b), A = 0, and A, = by. 
Basis 
A collection of vectors a,,a5,...,a, forms a basis of R” if the following 
conditions hold: 

1. aj,a9,...,a, span R”. 

2. Ifany of these vectors is deleted, the remaining collection of vectors 


does not span R”. 
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It can be shown that the foregoing conditions are equivalent to the following 
two requirements: k = and a),4>,...,a,, are linearly independent. To illustrate, 


1 0 
consider the two vectors a; = () and ay = () in R?. These two vectors 


form a basis of R? since k= n= 2, and a, and a, are linearly independent. 
Given a basis of R”, say a),@,...,a,, any vector b in R” is uniquely 


represented in terms of this basis. If b = pei Ajj and also b = "al Aja j, 


then Va; — A;)a; =0, which implies thatA; = 4; for each j, since 
otherwise, we would violate the linear independence of a),45,...,a,.- 


Since a basis in R” must always have n vectors, then the dimension of a 
basis is unique, namely, v. But a basis itself is not unique, since any set of 


vectors that are linearly independent will form a basis in R”. 
Replacing a Vector in the Basis by Another Vector 


In the simplex method discussed in Chapter 3, different bases will be generated, 
where one vector from the last basis is replaced by another vector. We have to 
be careful in choosing the vectors entering and leaving the basis, because 
otherwise, the new vectors may not be linearly independent, and hence will not 
form a basis. To illustrate, the vectors a, = (1, 2,1), a2 = (3, 0,1), and a3 = 


(2, -2, 1) are linearly independent, and hence form a basis of R?. We cannot 
replace a3 by (2, —2, 0), because a), ay, and (2, —2, 0) are linearly dependent 
and do not form a basis. 

This leads to the following natural question: If we have a basis of R”, 
what is the condition that will guarantee that if a vector of the basis, say a p 1S 
replaced by another vector, say a, then the new set of vectors still forms a basis? 

Let a;,@,...,a, form a basis of R”. We want to replace a, by a. Since 


a,,...,a, form a basis, then a can be represented as a linear combination of these 
vectors, that is, 


Suppose that 2; # 0. We shall show that the vectors aj, a9,...,a;_1, 
a, @j4),..-,@, are linearly independent, and hence form a basis. Suppose that 


there exist 4 and yw;(i ¥ /), such that 


dX “4a; + wa = 0. 


it] 
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Substituting a = )7_,4;a;, we get 


An 
» ja; + uy Aa; a 0, 
pan i=] 


it] i 
Le, LY (uj + HA)a; + wAja; = 0. 
iF] 


But since A), .A2,...,A;,..,8, are linearly independent, then BA; = 0, and 
H; + HA, = 0 for i # j. Since A; # 0 by assumption, then y = 0. But this 
implies that 4“; = 0 for i # j. In other words, ¥j, ;4;a; + “a = 0 is only 
possible if 4 = 0 and yw; = 0 for i # j, and hence, a and a; for i ¥j are 


linearly independent and must form a basis. From this discussion it is obvious 
that the condition A; # 0 is sufficient for the new set of vectors to be linearly 


independent. Obviously, the condition is also necessary, because if A; were 
zero, then a — Yj, ;A;a; = 0, and hence, a and a, for i # j would be linearly 


dependent. 


2.2 MATRICES 


A matrix is a rectangular array of numbers. If the matrix has m rows and n 
columns, it is called an m x n matrix (read “m by n”). Matrices will be denoted 
by capital boldface letters, such as A, B, C. An example of a 3 x 2 matrix is 


given below: 
1 -l 
A=;2 2). 
3 1 


The entry in row i and column j is denoted by a; for example, a2 = —1 and 


a3, = 3. Anm x n matrix A can be represented by its columns or by its rows. If 
we denote the columns of A by aj,49,...,a,,, then A = [a),4,...,a, ]. Similarly, 
A can be represented as 


where al, a’,...,a” are the rows of A. Note that every vector is a matrix, but 
every matrix is not necessarily a vector. 
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Addition of Matrices 


The addition of two matrices of the same dimension is defined component—wise; 
that is, if A and B are m x n matrices, then C = A + B is defined by letting 
Cy = ay + by, for i= 1,..., mand j= 1,..., 7. 

Multiplication by a Scalar 

Let A be an m x n matrix and let & be a scalar. Then kA is an m x n matrix 
whose jj entry is kaj. 

Matrix Multiplication 


Let A be an m x n matrix and B be an » x p matrix. Then the product AB is 
defined to be the m x p matrix C with 


n 
Cy = a for i=1.,...,m and J= 1.05 Dp. 


In other words, the ij entry of C is determined as the inner product of the ith row 
of A and the jth column of B. Let 


1 -1 1 5 0 
A=/4 -—2 5| and B=/}3 0}. 
2 0 1 [> +] 


1 -1 1)//5 0 3 1 
C=AB=/4 —2 5]/3 O0/=/19 S|. 
2 O lf{l 1 1] 1 


The following points need to be emphasized. If A is an m x m matrix and B is a 
p <q matrix, then: 


Then 


1. AB is defined only ifn =p. AB is then an m x qg matrix. 

2. BA is defined only if g =m. BA is then ap x matrix. 

3. Even if AB and BA are both defined (if m =n = p = q), then AB is 
not necessarily equal to BA. Note that AB in the foregoing example 
is defined, but BA is not defined. 


Special Matrices 


ZERO MATRIX 
An m x n matrix is called the zero matrix if each entry in the matrix is zero. 
IDENTITY MATRIX 


A square m x n matrix is called the identity matrix, denoted by I (sometimes the 
notation I,, is used to denote its size), if it has entries equal to one on the 
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diagonal and zero entries everywhere else. Note that AI, = A and I,,A=A for 
any m X n matrix A. 


TRIANGULAR MATRIX 


A square ” < n matrix is called an upper triangular matrix if all the entries 
below the diagonal are zeros. Similarly, an m x n matrix is called a lower 
triangular matrix if all elements above the diagonal are zeros. 


Transposition 
Given an m x n matrix A with ay as its ij entry, the transpose of A, denoted by 
A‘, is ann x m matrix whose jj entry is a;;. In other words, A’ is formed by 
letting the jth column of A be the jth row of A’ (similarly, by letting the jth row 
of A be the jth column of A‘). A square matrix A is called symmetric if A= A‘ 
and skew-symmetric if A= —A‘. The following results are obvious: 

1 (A‘) =A. 

2. IfA and B have the same dimension, then (A + B)’ = A’ + B’. 

3. If AB is defined, then (AB)’ = B’A‘. 


Partitioned Matrices 


Given an m x n matrix A, we can obtain a submatrix of A by deleting certain 
rows and/or columns of A. Hence, we can think of a matrix A as being 
partitioned into submatrices. For example, consider 


Here A has been partitioned into four submatrices: A,;, Aj, A>;, and Ay); 
A A 
aes eu 12 
Aa, | Ago 


a1 2 43° a4 
Ai; =| 421 2], Ariz =| 43 a4], 


therefore 


where 


43; 432 433 934 
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Az; = [44; 42], Azz = [443 aq]. 
Now, suppose that A and B are partitioned as follows: 
mi ny 1 #9 4 
ro Pt |B Pies Bi | Bi | Ba | 2 
Aa; | A22 | m Bo; | B22 | Baz] po. 
The AB is defined by 
A A B B B 
soe Oe Bi eB 
Aa | A22 || Bo; | Baz | Bos 
— | AB + AiBo) | AiBig + Ai2Bo2 | ArBi3 + Ai2Bo3 
A By); + Az2Bo) | Az)Byz + Az2By2 | A>)B)3 + A22Bo3 |’ 


Note that we must have nm, = p, and ny = po, so that the product of the sub- 
matrices is well defined. 


Elementary Matrix Operations 


Given an m X n matrix A, we can perform some elementary row and column 
operations. These operations are most helpful in solving a system of linear 
equations and in finding the inverse of a matrix (to be defined later). 


An elementary row operation on a matrix A is one of the following 
operations: 

1. Row iand row/ of A are interchanged. 

2. Row iis multiplied by a nonzero scalar k. 

3. Row is replaced by row i plus k times row /. 


Elementary row operations on a matrix A are equivalent to premultiplying A by 
a specific matrix. Elementary column operations are defined similarly. 
Elementary column operations on A are equivalent to postmultiplying A by a 
specific matrix. 


Example 2.1 
Let 


We shall perform the following elementary operations on A. Divide row | by 2, 
then add the new row | to row 2 and subtract it from row 3. This gives 


1 2 2 5 
0 5/2 3/2 13]. 
0) =3/2- 3/2 3 
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Now, multiply row 2 by 2/5, then multiply the new row 2 by 3/2 and add it to 
row 3. This gives 

io: 3» . =5 

0 1 3/5 26/5. 

0 0 24/10 24/5 


Divide row 3 by 24/10. This gives 


1/2 2 #5 
0 1 3/5 26/5]. 
O° T° 2 


Note that the matrix A is reduced to the foregoing matrix through elementary 
row operations. In particular, the first operation is performed by premultiplying 


A by the following matrix: 
/2 0 0 
1/2 1 O}. 


=1/9°0.<4 


Solving a System of Linear Equations by Elementary Matrix Operations 
Consider the system Ax = b of m equations in » unknowns, where A is an m x n 
matrix, b is an m-vector, and x is an n-vector of variables. The following fact is 
helpful in solving this system: Ax = b if and only if A’x = b’, where (A‘, b’) is 
obtained from (A, b) by a finite number of elementary row operations. To 
illustrate, consider the following system: 


2x, + XX + x3 = 10 
—X} + 2X, + »% = 8 
xX Xp + 2x3 = 2: 


Hence, 


This matrix was reduced in Example 2.1 through elementary row operations to 


: "Ys 23 
0 0 l 2 


Therefore, x solves the original system if and only if it solves the 
following system: 


x + (1/2)x. + (1/2)x3 
xy + (3/5)x3 


5 
26/5 
x3 23 
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Note that A’ is upper triangular, and we can solve the system by back- 
substitution. From the third equation x3; = 2, from the second equation 


X_ = 4, and from the first equation x, = 2. The process of reducing A into an 


upper triangular matrix with ones on the diagonal is called Gaussian reduction 
of the system. By performing further row operations, we could have reduced A 
into an identity matrix, from which the vector b would have been transformed to 


(2,4,2)'. This process is called a Gauss—Jordan reduction of the system. 


Matrix Inversion 


Let A be a square » x n matrix. If B is an x m matrix such that AB = I and BA 
= J, then B is called the inverse of A. The inverse matrix, if it exists, is unique 


and is denoted by A7!. If A has an inverse, A is called nonsingular; otherwise, 
A is called singular. 
CONDITION FOR EXISTENCE OF THE INVERSE 


Given an 7” X n matrix A, it has an inverse if and only if the rows of A are 
linearly independent or, equivalently, if the columns of A are linearly 
independent. 


CALCULATION OF THE INVERSE 


The inverse matrix, if it exists, can be obtained through a finite number of 
elementary row operations. This can be done by noting that if a sequence of ele— 
mentary row operations reduce A to the identity, then the same sequence of 


operations will reduce (A,I) to (LA). In fact, this is equivalent to pre— 
multiplying the system by A’! Further, if (A,B) is reduced to (IF) by 


elementary row operations, then F = A|B. 
In order to calculate the inverse, we adjoin the identity to A. We then 
reduce the matrix A to the identity matrix using elementary row operations. This 


will result in reducing the identity to A of course, if A! does not exist, then 


the elementary row operations will fail to produce the identity. This discussion 
is made clear by the following two examples. 


Example 2.2 
(At exists) 


Consider the matrix A: 


2 1 1 
A=/-l 2 1}. 
1 -1 2 


To find the inverse, form the augmented matrix (A, I) and reduce A by 
elementary row operations to the identity. The matrix in place of I will then be 


Al. To begin: 
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2 1 1/1 0 0 
-1 2 1/0 1 O|. 
1-1 2)/0 0 1 


Divide the first row by 2. Add the new first row to the second row and subtract 
it from the third row: 


f V2 2{ 1/2 0 j 
0 5/2 3/2} 1/2 1 Ol. 
0 -3/2 3/2}-1/2 0 1 


Multiply the second row by 2/5. Multiply the new second row by —1/2 and 
add to the first row. Multiply the new second row by 3/2 and add to the third 


: 0 5] 2/5 -1/5 j 
0 1 3/5| 5 2/5 O}. 
0 0 12/5|-1/5 3/5 1 


Multiply the third row by 5/12. Multiply the new third row by —3/5 and add to 
the second row. Multiply the new third row by —1/5 and add to the first row: 


10 0] 5/12 -3/12 -1/12 
0 1 0] 3/12 3/12 -3/12|. 
0 0 1}-1/12 3/12 5/12 


Therefore, the inverse of A exists and is given by 
, (ules Soe Sl 
A =—| 3 3 -3). 
-l1 3 #5 
Example 2.3 


(Al does not exist) 


Consider the matrix A: 


1 12 
A=|2 -1 1}. 
1 2 3 


The inverse does not exist since a3 = a; + a). If we use the foregoing pro— 
cedure, the elementary matrix operations will fail to produce the identity. To 


begin, we form: 
1 1 2)1 
2 -1 1/0 
1 2 340 
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Multiply the first row by —2 and add to the second row. Multiply the first row 
by —1 and add to the third row: 
10 0 
—2 1 Ol. 


1 1 2 
0 -3 -3 
-1 0 1 


0 1 1 


Multiply the second row by —1/3. Then multiply the new second row by —1 and 
add to the first row. Multiply the new second row by —1 and add to the third 


row: 
10 1] 1/3 13 0 
0 1 1} 2/3 -1/3 01. 
00 0{-5/ 1/3 1 


There is no way that the left-hand-side matrix can be transformed into the 
identity matrix by elementary row operations, and hence, the matrix A has no 
inverse. 

The following facts about matrix inversion are useful: 


1. If A is nonsingular, then A’ is also nonsingular and (A‘)! = 
(Avy! 

2. If A and B are both nm x nv” nonsingular matrices, then AB is 
nonsingular and (AB)! =BlAT 

3. A triangular matrix (either lower or upper triangular) with nonzero 
diagonal elements has an inverse. This can be easily established by 
noting that such a matrix can be reduced to the identity by a finite 


number of elementary row operations. In particular, let D = 
diag {d,,...,d,,} be a diagonal matrix with diagonal elements 


d,,....d, and all other elements equal to zero. If d,...,d, are all 
nonzero, then D7! = diag{1/dj,...,1/d,}. 


4. Let A be partitioned as follows, where D is nonsingular: 


mh M2 


LE] 1 
~|0)D] wn. 


Then A is nonsingular and 
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Determinant of a Matrix 
Associated with each square n x n matrix is a real number called the 
determinant of the matrix. Let A be ann x n matrix whose ij element is ay. The 


determinant of A, denoted by det A, is defined as follows: 
n 
det A = > a; Aj, 
i=l 


where 4,, is the cofactor of a; defined as ey times the determinant of the 


submatrix of A obtained by deleting the ith row and first column. The 
determinant of a 1 x | matrix is just the element itself. To illustrate, consider the 
following example: 


10 1] 
det} 2 1 -3]= 14), + 24> = 3A] 
—3 2 1 


s ] -3 0 1 | 
= det} | - 2aee| 8 1] - 3aee|9 5 


Note that the foregoing definition reduces the calculation of a determinant of an 
n X n matrix to n determinants of (7 —1)x(n-—1) matrices. The same 
definition can be used to reduce the determinant of an (” — 1) x (n — 1) matrix 
to determinants of (n — 2) x (m — 2) matrices, and so forth. Obviously, by this 
a] 


definition, the determinant of a 2 x 2 matrix, say A’ = 2 
21 


al is simply 
422 
1922 — 421412. 

To summarize, the determinant of an n < n matrix can be calculated by 
successively applying the foregoing definition. The determinant of A is 
therefore given by 1(1 + 6) — 2(0 — 2) — 3(0 — 1) = 14. We summarize some 
important facts about determinants of square matrices. 


1. In the definition, the first column was used as a reference in 


calculating det A. Any column or row can be used as a reference in 
the calculations, that is, 


detA = >> ay Ay, for any j = 1,..., 7, 


and similarly, 


n 
detA = >} a,A,, foranyi=1,..., 7, 
j= 
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where 4; is the cofactor of a, given as (-1)'*4 times the deter— 


minant of the submatrix obtained from A by deleting the ith row and 
jth column. 

det A = det A‘. 

Let B be obtained from A by interchanging two rows (or columns). 
Then det B = —det A. 

Let B be obtained from A by adding to one row (column) a constant 
times another row (column). Then det B = det A. 

Let B be obtained from A by multiplying a row (or column) by a 
scalar k. Then detB = kdet A. 

Let A be partitioned as follows, where B and C are square: 


Then det A = det B- det C. 


By a repeated application of Fact 6, the determinant of a triangular 
matrix is simply the product of the diagonal entries. 
Let A and B be x x 1 matrices. Then det(AB) = det A - det B. 


det A # 0 if and only if the columns (and rows) of A are linearly 
independent. Equivalently, det A = 0 if and only if the rows 
(columns) of A are linearly dependent. Therefore, a square matrix A 
has an inverse if and only if its determinant is not zero. 


. Let A be ann  n matrix whose determinant is not zero. Then AW! 


exists and is given by 


re -B 
detA 


where B is the transpose of the matrix whose jj entry is Aj, the 
cofactor of a,;. Here B is called the adjoint matrix of A. 


Consider the system Ax = b where A is 7 < n, b is an -vector, and x 
is an n-vector of unknowns. If A has an inverse (that is, if 
det A # 0), then the unique solution to this system is given by 


det A; i 
x; = , forj=1,...,7, 
: det A qf 


where A; is obtained from A by replacing the jth column of A by b. 


This method for solving the system of equations is called Cramer’s 
Rule. 
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The Rank of a Matrix 


Let A be an m x n matrix. The row rank of the matrix is equal to the maximum 
number of linearly independent rows of A. The column rank of A is the 
maximum number of linearly independent columns of A. 

It can be shown that the row rank of a matrix is always equal to its 
column rank, and hence the rank of the matrix is equal to the maximum number 
of linearly independent rows (or columns) of A. Thus it is clear that rank (A) < 
minimum {m, n}. If rank (A) = minimum {m, n}, A is said to be of full rank. It 
can be shown that the rank of A is k if and only if A can be reduced to 


through a finite sequence of elementary matrix operations. 
2.3 SIMULTANEOUS LINEAR EQUATIONS 


Consider the system Ax = b and the augmented matrix (A, b) with m rows and n 
+ 1 columns. If the rank of (A, b) is greater than the rank of A, then b cannot be 


represented as a linear combination of a),45,...,a,, and hence there is no 


solution to the system Ax = b (and in particular, there is no solution to the 
system Ax =b,x 2 0). 

Now, let us suppose that rank (A) = rank (A, b) = &. Possibly after 
rearranging the rows of (A, b), let 


A; b, 
(A, b) = , 
Ar bp 
where A, is k x n, b, is a k-vector, Ay is an (m—k) xn matrix, by is an 
(m — k)-vector, and rank (A, ) = rank (A, ,b, ) =. 
Note that if a vector x satisfies A,x = b,, then it satisfies A,x = b> 


automatically. Therefore, we can throw away the “redundant” or “dependent” 
constraints A,x = b, and keep the independent constraints A,x = b;. Since 


rank (A, ) = k, we can pick k linearly independent columns of A,. Possibly 
after rearranging the columns of A,, let A, = (B, N), where B is ak x k 
nonsingular matrix and N is k x (n — k). Note that such a matrix B exists since 
A, has rank k. Here B is called a basis matrix (since the columns of B form a 


basis of R* ) and N is called the corresponding nonbasic matrix. Let us 
decompose x accordingly into xz, and xy, where xg is composed of 
X15X75---X, and xy is composed of x, ,,,...,x,. Now, A ,x = b, means that 


@.n)|* | = bj, that is, Bxg + Nxy = bj. Since B has an inverse, then we 
XN 


can solve for xp in terms of xy by premultiplying by B!, and we get 
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xp = Bb, — BNxy. 


In the case k = n, N is vacuous, and we have a unique solution to the system 
A,x= b), namely, xp = Bb, 7 A |b). On the other hand, if n > &, then by 


assigning arbitrary values to the vector x,y, we can correspondingly compute 
. ; = 3 ; ; x 
Xp via the equation xp = B 'b, -B INKy to obtain a solution Ey to the 
N 


system A,x = b,. In this case, we have an infinite number of solutions to the 
system A,x = b, (and hence to the system Ax = b). Note that the notion of 
decomposing A, into B and N and deriving xp = B'b, = BINxy can be 
interpreted as follows. We have a system of & equations in n unknowns. Assign 
arbitrary values to n — k of the variables, corresponding to x,y, and then solve 


for the remaining system of k equations in k unknowns. This is done such that 
the k equations in k unknowns have a unique solution, and that is why we 


require B to have an inverse. Such a solution obtained by letting x, = 0 and 
xz = B'b, is called a basic solution of the system A,x = by. Let us now 
summarize the different possible cases that may arise: 


1. Rank (A, b) > rank (A), and hence Ax = b has no solution. 

2. Rank (A, b) = rank (A) = & = n, and hence there exists a unique 
solution to the system Ax = b. 

3. Rank (A, b) = rank (A) = & < x, and hence we have an infinite 
number of solutions to the system Ax = b. 


In general, these cases may be determined through a Gauss—Jordan 
reduction of the system Ax = b to the following form A‘’x = b’ via elementary 


row Operations: 
I, Q _ bj 
0 0 b5 | 


Hence, rank (A) =k < min{m, n}. Ifk<mand b> # 0, we have Case (1). Ifk 
=n <m,and also b) = 0 when m>n, we have Case (2). Ifk <n, and also 
b) = 0 when k < m, we have Case (3). 

Example 2.4 


Consider the following system: 


x, + 2X oN or, 2X4 = 10 
—X) + 2X. - % + =X = 6 
XxX + X3 = 2. 


We shall solve this system by matrix inversion (Gauss—Jordan reduction) and 
also by only using a Gaussian reduction. 
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1. Matrix Inversion. Reduce three columns of A to the identity (this is 
possible since rank (A) = 3). To begin, we form: 


12 1 -2)10 
-1 2 -1 1] 6}. 
2 


Add the first row to the second row: 
1 2 1 21/10 

04 0 -1/16/. 

2 


0141 0 
Divide the second row by 4. Multiply the new second row by —2 and 
add to the first row. Multiply the new second row by —1 and add to 


the third row: 
10 1 -3/2{ 2 
0 10 -1/4} 4). 
—2 


001 1/4 
Multiply the third row by —1 and add to the first row: 


4 
4. 
—2 


The original system has been reduced to the system shown above. 
Equivalence of the two systems is assured since the new system is 
obtained from the original system after performing a finite number of 
elementary row operations. The solution to the system is as follows. 
Assign x4 arbitrarily, say x, = A. Then x, = 4+ (7/4)A, x = 
4+ (1/4)A, and x3 = -2 — (1/4)A. 

2. Gaussian Reduction. To begin, we form: 


12 1 2110 

-1 2 -l 1] 16). 

01 1 =O} 2 

Add the first row to the second row: 
12 1 2/10 

0 4 0 -1/) 16). 
2 


011 £40 
Divide the second row by 4. Subtract the new second row from the 
third row: 


xX XD XB X4 
1 0 0 -7/4 
0 1 0 -1/4 
0 0 1 14 
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10 
4|. 
23 


The foregoing matrix has an upper triangular submatrix. Let x4 be 


xX Xp 3 x4 


ide a ey 
0 1 0 -1/4 
O° SO. i afd 


equal to an arbitrary value A. Then x; = —2 - (1/4)A, x. =4 + 
(1/4)A, and x, = 10+ 2A - 2x, ~ x3 = 4+(7/4)A. This gives 
the same general solution obtained earlier, as expected. 

2.4 CONVEX SETS AND CONVEX FUNCTIONS 


In this section we consider some basic properties of convex sets, convex 
functions, and concave functions. 


Convex Sets 


A set Xin R” is called a convex set if given any two points x, and x, in X, 
then Ax, + (1 — 4)xy € X foreach A e [0,1]. 

Note that Ax, + (1 — 4)x2 for / in the interval [0, 1] represents a point 
on the line segment joining x, and x. Any point of the form Ax, + (1 — 4)x, 
where 0 < A < 1 is called a convex combination (or weighted average) of x, 
and x. If A e (0,1), then the convex combination is called strict. Hence, 
convexity of X can be interpreted geometrically as follows. For each pair of 
points x, and X» in X, the line segment joining them, or the convex 
combinations of the two points, must belong to_X. 

Figure 2.5 shows an example of a convex set and an example of a 
nonconvex set. In the latter case, we see that not all convex combinations of x, 
and x» belong to X. The following are some examples of convex sets: 

2 2 

1. {(%1,x2) 2 xy + x9 < }}. 

2. {x :Ax=b}, where A is an m x n matrix and b is an m-vector. 

3. {x : Ax = b, x = 0}, where A is an m x n matrix and b is an m- 

vector. 


4. {x : Ax < b,x > 0}, where A is an m x n matrix and b is an m- 
vector. 


1 1 -1 
fe x =| Hl +A, H val 3 av = 1,A,,4,,4; = 7 
0 1 3 


Extreme Points 


WG 


The notion of extreme points plays an especially important role in the theory of 
linear programming. A point x in a convex set X is called an extreme point of X 
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ie 


A convex set A nonconvex set 
Figure 2.5. Examples of convex and nonconvex sets. 


if x cannot be represented as a strict convex combination of two distinct points 
in X. In other words, if x = Ax, + (1 - 4)x2 with 4 e€ (0,1) and x,,xy € X, 
then Xx = X; = X». 

Figure 2.6 shows some examples of extreme and nonextreme points of 
convex sets. Note that x, is an extreme point of X, whereas x and x3 are not. 
More insights into extreme points are given in Section 2.6. 


Hyperplanes and Half—Spaces 


A hyperplane in R” generalizes the notion of a straight line in R* and the 
notion of a plane in RSA hyperplane H in R" isa set of the form {x : px = k} 


where p is a nonzero vector in R” and k is a scalar. Here, p is called the normal 
or the gradient to the hyperplane. 
Equivalently, a hyperplane consists of all points x = (x,%9,...,X,) 


satisfying the equation Lisl p;x; = k. The constant k can be eliminated by 


referring to a fixed point xp on the hyperplane. If xg ¢ H, then pxg =k, and 
for any x e H, we have px =k. Upon subtraction we get p(x — xp) = 0. In 


other words, H can be represented as the collection of points satisfying 
p(x — xq) =0, where xg is any fixed point in H. A hyperplane is a convex set. 


Nonextreme points __*2 


a Extreme point 
x] 


Figure 2.6. Extreme and nonextreme points. 
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Direction of p 


Hyperplane H 


Direction of x — x 


Figure 2.7. Hyperplane. 


Figure 2.7 shows a hyperplane and its normal vector p. Note that p is 
orthogonal to x — Xp for each x in the hyperplane H. A hyperplane divides R” 
into two regions, called half-spaces. Hence, a half-space is a collection of 
points of the form {x : px > k}, where p is a nonzero vector in R” and k isa 
scalar. A half-space can also be represented as a set of points of the form {x : 
px < k}. The union of the two half-spaces {x : px = k} and {x : px < k} is 
R". Referring to a fixed point xg in the hyperplane defining the two half— 
spaces the latter can be represented as {x : p(x — Xo) = 0} oras {x : p(x —Xq) 
< 0}, as shown in Figure 2.8. Note that the first half-space consists of points x 
for which (x —Xg) makes an acute angle (< 90°) with p, whereas the second 
half-space is comprised of points x such that (x — xq) makes an obtuse angle 
(2 90°) with p. 

Rays and Directions 

Another example of a convex set is a ray. A ray is a collection of points of the 
form {xq + Ad : A = 0}, where d is a nonzero vector. Here, Xq is called the 
vertex of the ray, and d is the direction of the ray. 

Directions of a Convex Set 

Given a convex set, a nonzero vector d is called (recession) direction of the set, 
if for each Xp in the set, the ray {xg + Ad : A = 0} also belongs to the set. 
Hence, starting at any point xq in the set, one can recede along d for any step 


length 2 > 0 and remain within the set. Clearly, if the set is bounded, then it 
has no directions. 

Consider the nonempty polyhedral set X = {x : Ax < b,x 2 0}. Then 
a nonzero d is a direction of X if and only if 


A(x + Ad) 
x + Ad 


b 
0 


VIA 
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Direction of p __— Hyperplane H 


Figure 2.8. Half—spaces. 


for each 2 > 0 and each x € X. Since x € X, then Ax < b and the first 
inequality holds for A = 0 arbitrarily large if and only if Ad < 0. Similarly, x 
+ Ad is nonnegative for 4 arbitrarily large, if and only if d is nonnegative. To 
summarize, d is a (recession) direction of X if and only if 


d > 0, dz0, and Ad < 0. 
Similarly, if X = {x : Ax = b,x = 0} # ©, then by replacing the equality by 


two inequalities, it follows that d is a direction of X if and only if d ¥ 0,d >0, 
and Ad = 0. The set of directions in either case forms a convex set. 


Example 2.5 

Consider the set X ={(x,,x2):x,—2x) 2-6, x1 — xX) 2-2, xm 20,x 2 1} 

depicted in Figure 2.9, Let xp = (3!) be an arbitrary fixed feasible point. Then 

d=(4% | isa direction of X if and only if “| )#(°) and xy +4d = 
d> yas)” \0 0 

belongs to X for all 4 = 0. Therefore, 


xy + Ad; 
x2 + Ad 


My - 2x, + A(d,-2d,) = -6 
X - XxX + A(dj-d,) = -2 
x + Ad, = 0 

XxX, + Ad» = 1 


for all A 2 0. Since the last two inequalities must hold for the fixed x, and x, 
and for all 4 2 0, we conclude that d; and dy > 0 (why?). Similarly, from 


the first two inequalities we conclude that d, - 2d, 20 and d, —-d, 20 
(why?). 


Since d, and d, 2 0, then d, = 2d, implies that d, = dy. Therefore, 


(3) is a direction of X if and only if 
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dz 
=. Directions of X 


dq) 


x 


Figure 2.9. Directions of convex sets. 


(4) + (6) 


d, > 2d. 


This collection of vectors is shown in Figure 2.9 and can be normalized such 
that each direction has norm (or length) equal to 1. 


EXTREME DIRECTIONS OF A CONVEX SET 


The notion of extreme directions is similar to the notion of extreme points. An 
extreme direction of a convex set is a direction of the set that cannot be 
represented as a positive combination of two distinct directions of the set. Two 


vectors, d,; and dy, are said to be distinct, or not equivalent, if d, cannot be 


represented as a positive multiple of dy. In the foregoing example, after 
normalization, we have two extreme directions d, = (1, 0) and d, = (2/ V5 , 
1//5). Any other direction of the set, which is not a multiple of d, or d5, can 


be represented as 7d, + A,d>, where 4,,2, >0. Any ray that is contained in 


the convex set and whose direction is an extreme direction is called an extreme 
ray. Section 2.6 provides further insights into recession directions and extreme 
directions. 


Convex Cones 


A special important class of convex sets is convex cones. A convex cone C is a 
convex set with the additional property that Ax € C for each x € C and for 
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Ongin Origin 


Figure 2.10. Some examples of convex cones. 


each A > 0. Note that a convex cone always contains the origin by letting 2 =0, 
and also, given any point x € C, the ray or half line {Ax : A = 0} belongs to 
C. Hence, a convex cone is a convex set that consists entirely of rays emanating 
from the origin. Figure 2.10 shows some examples of convex cones. Since a 
convex cone is formed by its rays, then a convex cone can be entirely 
characterized by its directions. In fact, not all directions are needed, since a non— 
extreme direction can be represented as a positive combination of extreme 
directions. In other words, a convex cone is fully characterized by its extreme 


directions. 
As an example, consider the convex cone whose extreme directions are 
(1, 1) and (0, 1). From Figure 2.11 it is clear that the convex cone must be the 


set {(%1,X) 2 4 2 0.x, < xX}. 

Given a set of vectors aj,a>,...,a;, we can form the convex cone C 
generated by these vectors. This cone consists of all nonnegative combinations 
of aj,a5,...,a;,, that is, 


k 
C= {2 Aja; . Aj 2 0, for j = th 
jal 


Figure 2.11 shows the convex cone generated by the vectors (0, 1) and (1, 1). 


d, = (0, 1) 


Figure 2.11. Characterization of convex cones in terms of 
extreme directions. 
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J I 
F(x) 
F(x) a) f(x) 
Ej 
x X2 Xx) X2 Xj X2 
(a) (b) (c) 


Figure 2.12. Examples of convex and concave functions: (a) Convex 
function. (b) Concave function. (c) Neither convex nor concave. 


Convex and Concave Functions 


Convex and concave functions play an important role in optimization problems. 
These functions naturally arise in linear optimization problems (in a nonlinear 
form) when dealing with parametric analysis. 

A function f of the vector of variables (x), x2,....X,) 1s said to be convex 


if the following inequality holds for any two vectors x, and x): 

S(Ax, + (1 — 4)x2) S$ AfS(x)) + (1 - Af (x2), for all 4 € [0,1]. 
Figure 2.12a shows an example of a convex function. Note that the foregoing 
inequality can be interpreted as follows: Af(x,) + (1 — 4)f(x2), where A € 
[0, 1], represents the height of the chord joining (x,, /(x,)) and (x, f(x2)) at 
the point Ax; + (1 — A)x2. Since Af(x,) + (l- AS (x2) 2 f(Ax, + i - A) 
X>), then the height of the chord is at least as large as the height of the function 
itself. 

A function f is concave if and only if —f is convex. This can be restated 
as follows: 


f(Ax, + (1 - Axa) => Af (x) + (1- AF (Kp), for all 2 € [0,1], 


for any given x, and x>. Figure 2.12b shows an example of a concave 
function. An example of a function that is neither convex nor concave is 
depicted in Figure 2.1 2c. 

2.5 POLYHEDRAL SETS AND POLYHEDRAL CONES 


Polyhedral sets and polyhedral cones represent important special cases of 
convex sets and convex cones. A polyhedral set or a polyhedron is the 
intersection of a finite number of half—-spaces. A bounded polyhedral set is 
called a polytope. Since a half-space can be represented by an inequality of the 
type alx < b;, then a polyhedral set can be represented by the system a'x < b; 


for i = 1,..., m. Hence, a polyhedral set can be represented by {x : Ax < b} 


where A is an m x matrix whose ith row is a’ and b is an m-vector. Since an 
equation can be written as two inequalities, a polyhedral set can be represented 
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by a finite number of linear inequalities and/or equations. As an example, 
consider the polyhedral set defined by the following inequalities: 


-2x, + xX < 4 
XH + xX < 3 

x < 2 

x 2 0 
xX 2 0. 


The intersection of these five half-spaces gives the shaded set of Figure 2.13. 
Clearly the set is a convex set. We can see a distinct difference between the first 
inequality and the remaining inequalities. If the first inequality is disregarded, 
the polyhedral set is not affected. Such an inequality is called (geometrically) 
redundant or irrelevant to the polyhedral set. 

A special class of polyhedral sets is the set of polyhedral cones. A 
polyhedral cone is the intersection of a finite number of half—spaces, whose 
hyperplanes pass through the origin. In other words, C is a polyhedral cone if it 
can be represented as {x : Ax < 0}, where A is an m x n matrix. Note that the 


ith row of the matrix A is the normal vector to the hyperplane defining the ith 
half-space. Figure 2.11 shows an example of a polyhedral cone. 


2.6 EXTREME POINTS, FACES, DIRECTIONS, AND EXTREME 
DIRECTIONS OF POLYHEDRAL SETS: GEOMETRIC INSIGHTS 


In Section 2.4 we discussed extreme points and directions of general convex 
sets, although we often used polyhedral sets for illustration. In this section, we 
will provide some geometric insights and equivalent definitions of extreme 
points and directions for polyhedral sets. We will assume that the polyhedral set 
under discussion in this section is of the form 


X = {x: Ax <b,x>0} (2.1) 


where A is m x n and b is an m-vector. As shown in Chapter 1, through suitable 
linear transformations, the feasible region of any linear programming problem 
can be put into this so-called canonical form. 


Extreme Points 


We have already seen that an extreme point of X is a point in X that cannot be 
made to lie in the interior of a line segment contained within X. We will now 
provide an equivalent geometric definition. Toward this end, consider the 
following simple but important observation. Let x € X and suppose that some 
constraint ax < £ of X is binding, or tight, or active, at X that is, ax = f. 


Suppose further that we can write X = Ax’ + (1 — 2)x’, where 0 < 2 < 1 and 
where x’ and x” € X. Then it must be also true that ax’ = # and ax” = £, 


that is, ax < # must be binding at x’ and at x” as well. This clearly follows 
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Figure 2.13. Polyhedral set. 


from the fact that ax’ < Gand ax” < B (since x’, x” € X), but A(ax’) + 
(1 — A\(ax") = B (since ax = #), where 0 <A< 1. 

An extreme point can now be defined as follows. Let the hyperplanes 
associated with the (m + n) defining half-spaces of X be referred to as defining 
hyperplanes of X. Furthermore, note that a set of defining hyperplanes are 
linearly independent if the coefficient matrix associated with this set of 
equations has full row rank. Then a point x € X is said to be an extreme point 
or a corner point, or a vertex of X if X lies on some »# linearly independent 
defining hyperplanes of X. If more than n defining hyperplanes pass through an 
extreme point, then such an extreme point is called a degenerate extreme point. 
The excess number of planes over 7 is called its order of degeneracy. A 
polyhedron with at least one degenerate extreme point is said to be a degenerate 
polyhedral set. 

This definition of extreme points is readily seen to be equivalent to 
saying that X¥ cannot be written as a strict convex combination of two distinct 
points in X. When x € X lies on some n linearly independent defining 
hyperplanes, then if we can write X = Ax’ + (1 — 2)x”, where 0 < A <1 and x’, 
x” < X, we have that both x’ and x” also lie on these » hyperplanes. The 
solution to these ” hyperplane equations, however, is unique. Hence, x = 
x’ = x”. Conversely, if the maximum number of linearly independent defining 
hyperplanes binding at x € X are r <n and are given by Gx = g, where G is r 
x n, let d # 0 be a solution to Gd = 0. Note that such a d exists (why?). Then 
there exists an € > 0 such that x’ = (x + €d) e X and x” =(x-edje X, 
since Gx’ = g, Gx” = g and the constraints of X that are nonbinding at x 
remain satisfied provided ¢ > 0 is small enough. Consequently, since x = 
0.5x' + 0.5x” and x’, x” © X, we have that x is not an extreme point of X. 


Referring to the polyhedral set in Figure 2.14, note how every extreme 
point is formed by some » = 3 linearly independent defining hyperplanes, and at 
every other point, fewer than n = 3 linearly independent defining hyperplanes 
are binding. Note also that at the indicated degenerate extreme point, four 
defining hyperplanes are binding. The order of degeneracy is one. In particular, 
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Degenerate extreme point 


Two-dimensional face 


Adjacent extreme points 


Edge 


Figure 2.14. Extreme points and edges of X. 


there is more than one choice of selecting n = 3 linearly independent 
hyperplanes that give this vertex as their unique intersection point. However, no 
defining hyperplanes or half-space can be omitted without changing the 
structure of this polyhedron. That is, there are no redundant constraints. 


Faces, Edges, and Adjacent Extreme Points 


We have seen that each extreme point x of X is a unique solution to some n 
linearly independent defining hyperplanes binding at x. In general, the set of 
points in X that correspond to some nonempty set of binding defining 
hyperplanes of X is called a face of X. Now, given any face F of X, if r(F) is the 
maximum number of linearly independent defining hyperplanes that are binding 
at all points feasible to F, then the dimension of F, denoted dim(F), is equal to 
n — r(F) . In other words, each linearly independent binding hyperplane results 
in the loss of one “degree of freedom.” Consequently, extreme points are zero— 
dimensional faces of X. Similarly, an edge of X is a one-dimensional face of X, 
that is, it is the set of points in X formed by the intersection of some (n — 1) 
linearly independent defining hyperplanes (with some points in this set not 
having more than (” — 1) binding linearly independent hyperplanes). That is, an 
edge has one “degree of freedom” imparted by having one less than x linearly 
independent hyperplanes that are binding at all points. Similarly, if r(X) is 


defined with respect to X itself, then although X is in R”, it is actually of 
dimension dim(X) = n — r(X). The set X is said to be full dimensional if 
r(X) = 0, that is, dim(X) =n. Furthermore, the set Y (when not full 


dimensional or viewed in a higher dimension) and the empty set are also 
sometimes called improper faces of X itself. The other faces are called proper 
faces. The highest dimensional proper face of X is of dimension dim(X) —- 1 
(when dim(X) 2 1) and is called a facet of X. 

Finally, two extreme points of X are said to be adjacent if the line 
segment joining them is an edge of X. Hence, adjacent extreme points have 


some (n — 1) common binding linearly independent defining hyperplanes. Figure 
2.14 illustrates these definitions. 
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Figure 2.15. Directions and extreme directions. 


Recession Directions and Extreme Directions 


In Section 2.4, we characterized the directions of X as vectors d satisfying the 
conditions 


d 2 0, d + 0, and Ad < 0. 


Geometrically, this so—called homogeneous system defines a polyhedral cone 
(excepting the origin), also known as a recession cone, and is obtained by 
translating the defining hyperplanes of X parallel to themselves until they pass 
through the origin. To eliminate duplication, these directions may be 
normalized. To maintain linearity, it is convenient to normalize the directions 


using the (rectilinear) norm |d,| + --- + |d,| = 1, which leads to the normalization 


constraint d, + --- +d, =1, since d 2 0. Hence, the set 
D = {d: Ad < 0, 1d = 1, d 2 0} (2.2) 


characterizes the set of recession directions of X, where 1 is a vector of ones. 
Figure 2.15 illustrates this set. In fact, as seen in the figure, the extreme 
directions of X are precisely the extreme points of D. Clearly, because of the 
normalization constraint, a direction d € D cannot be written as a positive 
linear combination of two distinct directions if and only if it cannot be written as 
a strict convex combination of two distinct directions in D. 


Example 2.6 
Consider the polyhedral set X given by the inequalities 
3x, + x < -2 
—y + &% S$ 2 
| + 2X5 < 8 
- xX < 2 
X, xX 2 0. 
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x24 dy t 


Figure 2.16. Numerical example. 


The set is illustrated in Figure 2.16. Its extreme points are given as: 


x; = el X) = al and x3 = esl 


The set D of Equation (2.2) is given by 
D = {(d,,d2) : -3d, + dy < 0,-d, + dy < 0, 
-d, + 2d < 0,-d < 0, a; + dy = 1,d, 2 0,d> 2 O}. 


This set is sketched in Figure 2.16 and has extreme points d) = kal and 


d, = EI These are the two extreme directions of X. 


2.7 REPRESENTATION OF POLYHEDRAL SETS 


In this section, we discuss the representation of a polyhedral set in terms of 
extreme points and extreme directions. This alternative representation will prove 
very useful throughout the book. The proof of the representation theorem given 
here is insightful and instructive. The reader may want to study this proof in 
order to develop more confidence with the notions of extreme points and 
extreme directions. 


Basic Ideas 
1. Bounded Polyhedral Sets (Polytopes) 


Consider the bounded polyhedral set of Figure 2.17 (recall that a set is bounded 
if there is a number & such that ||x|| < & for each point x in the set), which is 
formed as the intersection of five half-spaces. We have five extreme points, 
labeled x),x2,x3,X4, and x5. Note that any point in the set can be represented 
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as a convex combination, or a weighted average, of these five extreme points. 
To illustrate, choose the point x shown in Figure 2.17. Note that x can be 
represented as a convex combination of y and x4, that is, 


x = Ay+(l1—A)x4, where A € (0,1). 
But y can itself be represented as a convex combination of x; and x, that is, 


y = wx, + (1-— 4)x., where we (0,1). 
Substituting, we get 
x = Aux, + A(L— w)xy + (1 - A)xg. 


Since A € (0,1) and w e (0,1), then Av, A(1—- yw), and (l1- A) e (0,1). 
Also, Au + ACL - yz) + (1 — A) = 1. In other words, x has been represented as a 
convex combination of the extreme points x;, x2, and x4. In general, any point 


in a bounded polyhedral set can be represented as a convex combination of its 
extreme points. (This is formally established by Theorem 2.1 below.) 


2. Unbounded Polyhedral Sets 
Let us now consider the case of an unbounded polyhedral set. An example is 
shown in Figure 2.18. We see that the set has three extreme points x,, x2, and 


X3, as well as two extreme directions d, and dy. From Figure 2.18 it is clear 


that, in general, we can represent every point in the set as a convex combination 
of the extreme points, plus a nonnegative linear combination of the extreme 
directions. To illustrate, consider the point x in Figure 2.18. The point x can be 
represented as y plus a positive multiple of the extreme direction d,. Note that 


the vector x—y points in the direction d,. But y itself is a convex 
combination of the extreme points x, and x3, and hence, 


Y= y + ud, 
AX, oe dl = A)x3 2 ud, 


x4 


x3 


x 
x2 : 
y x 


Figure 2.17. Representation in terms of extreme points. 
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Figure 2.18. Representation of polyhedral sets in terms of extreme points 
and extreme directions. 


where 4 € (0,1) and uw > 0. This discussion is made more precise later. 


We now state and prove the Representation/Resolution/Caratheodory 
Theorem for the general case of bounded or unbounded sets X. Figure 2.19 is 
provided to help conceptualize the ideas of the proof. 


Theorem 2.1 Representation Theorem for the General Case 


Let X= {x : Ax < b,x = 0} be a nonempty (polyhedral) set. Then the set of 
extreme points is nonempty and has a finite number of elements, say 
X},X2,....X;- Furthermore, the set of extreme directions is empty if and only if 
X is bounded. If X is not bounded, then the set of extreme directions is 
nonempty and has a finite number of elements, say d,,d4,...,dy. Moreover, 


x € X if and only if it can be represented as a convex combination of xj,...,X; 
plus a nonnegative linear combination of dj,...,d,, that is, 


k £ 
x = LX Ax; + dX wd; 
j=l j=l 


ai (2.3) 
j= 


= 
VW 
o Oo 
KON. 
ll ll 
Ss Oe 


Proof: 


For convenience, denote by S p and Sz the sets of extreme points and extreme 


directions of X. Let us first show that 1 < k < 0, where Is k. Toward this 


z| - 
end, let xe X. If X © Sp; then k = 1. Otherwise, let r be the maximum 


number of linearly independent defining hyperplanes binding at x and let 
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Polyhedron X with bounding hyperplane 


Xg =X3 + Ad) 


1x< M [Bounding hyperplane} 


5 
X> X= )X; + OpXy + O3Xz + OyXg 
¥2=%+7>(%-Y)) 1.€., X =O)X) +.0)Xq +(d; +d); + Ad, 


where 6, +6,+6,;+=1, 520, &>0. 


Figure 2.19. Illustration for the proof of Theorem 2.1. 


X=Ay,+(1-A)y2, where 0<A<1 and yj,y2 € X, y; # yy. Note that 
O<r<n. Let d=y2-y, #0, so that y;=x—(1-A)d and y, =x+Ad. Consider 
moving from x along d and —d. Both directions permit positive step lengths 
from the foregoing statement, but both cannot permit infinite step lengths since 
X C{x:x2>0}. Hence, without loss of generality, let us say that 
y =max{y:x—yde X}<oo and put y; =x — 7d. Note that the maximum num- 
ber of hyperplanes binding at y, must be yr >7r +1. This follows since the 
defining hyperplanes binding at x are also binding at y, (because xX is a strict 
convex combination of y; and yz). Moreover, at least one additional linearly 
independent hyperplane must be binding at y, to block any further motion 
along —d and hence determine 7. If 7 =n, then y; eS p? and so k > 1. Other- 


wise, we can replace x by y, and repeat this process until we obtain F =n, 
which, clearly, must occur finitely. Hence, k > 1. Because the number of ways n 
linearly independent hyperplanes can be chosen from (m + x) hyperplanes is 
finite, we also have k < oo. 

Next, by definition, if X is bounded, then D = @, and conversely, if X is 
unbounded, then D # ©, since X is a convex set. Noting that D is of the same 
form as X (where the equality in Equation (2.2) can be equivalently written as 
two inequalities) and that the extreme points of D correspond to the extreme 
directions of X, we have that 1 < @ < w, 

Now, suppose that X can be written as in Equation (2.3). Then it is easily 
verified using Equations (2.1) and (2.2) that x e X. Conversely, consider any 
x € X, and let us show that x can be written as in Equation (2.3). Define the 
set 
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X =X {x:1x < M} 
where M is large enough so that 1x; < M for each j= 1,.., 4, and Ix < M. 
Note that X is bounded, and moreover, extreme points of X are also extreme 
points of X. Let S,, = {Xj,...X,5--.X;4,} be the extreme points of X, where 
0 <u < oo. Let us first show that we can write x as a convex combination of 
points in ne If xe S53 this is obviously true. Otherwise, let the system Gx = 
g represent the hyperplanes of Y that are binding at x, and note that rank (G) 
<(a-1). Find a solution d #0 to the system Gd = 0, and compute 
yi =max{y:x+ydeX}. Note that 0 < 7, < «© (why?). Denote y,; =x + yd. 
Hence, at y, € X, we have at least one additional linearly independent 
hyperplane of Y binding. If the new binding hyperplane(s) along with Gx = g 
produce a system of rank n, then y, is a vertex X. Otherwise, we may repeat 
this step at y, until after at most [” — rank(G)] such steps, we obtain a vertex 
y, of X satisfying Gy, =g. Now, define 
7, = max{y :* + 7(K-Y)) € X} 
and put 
Y2 =X + 72(x — yj). 

(See Figure 2.19.) Observe that 7 <0 since X is bounded. Also, ¥2> 0, 
since G[x + y(x — y,)] =g for all y = 0 implies that 7, is determined by 
some constraint not binding at x. Hence, in particular, Gy, = g, and at least 
one additional linearly independent hyperplane is binding at y,. Furthermore, 
x is a convex combination dy, + (1-6)y, of y, and y>, where 
0 = ¥,/(1 + 72). Now, since y| € Sos if we also have y> € Sys then we 
have written x as a convex combination of points in S Pp Otherwise, we can 
write y, itself as a strict convex combination of some y3 and y4, where 
y,«S p and y4 has at least one additional linearly independent hyperplane 
binding. Continuing this way, we can ultimately write x as a convex 
combination of vertices of S,, in fact, using no more than n — rank(G) + 1 


vertices of S p: Let this representation be given by 


+u k+u 
> 6 Xj; where > 6; =], and 620. (2.4) 
j=l j=l 


k 
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Now, if 6; = 0 for j > &, then Equation (2.4) is of the required form of 
Equation (2.3). Otherwise, consider any x, with vy > and 6, > 0. Note that 


X, 1S anew extreme point created by the constraint 1x < M, that is, lx = M 


v 
is one of any 7 linearly independent hyperplanes defining x, in Y. The other 


(n — 1) linearly independent hyperplanes come from the defining hyperplanes of 
X, and therefore define an edge of X. Consequently, there exists some extreme 


point x;,) of X, 1 < i(v) < k, which is the adjacent extreme point of x, in xX 
along this edge. Moreover, (x, — X,,)) is a recession direction of X since no 
other hyperplane of X stops the progress along this direction from x,(,). More 
impor-tantly, let d = (x, — Xi(y))/,, where 0, =1(x, —X;(,))>0. Observe that 
d e¢ D. Furthermore, the (n — 1) linearly independent hyperplanes of the 
homogeneous system Ad < 0, d = 0, which correspond to the (m — 1) linearly 
independent hyperplanes of Y definingx,, are binding atd. Also, from the 
hyperplanes defining x,, these (n — 1) hyperplanes along with 1d = 1 produce 


n linearly independent binding hyperplanes of D at d. Hence, the direction d 
must be an extreme point d j(,), say, of D and therefore, an extreme direction of 


X. Consequently, we have x, = xjy) + 4d ;(,). Substituting this into Equation 


(2.4) for each such y, and arbitrarily letting i(v) = j(v) = 1 if 6, =0, we get 


= k k+u k+u 
x= 2 6X; + > OyXi(y) + > 5, O,4 jy); 
j=l v=k+1 v=k+1 


which is of the form Equation (2.3). This completes the proof. 
Corollary 2.1 


Any x e€ X can be represented as in Equation (2.3) using no more than min 
{(n + 1),(k + £)} positive 2;- and yi; -variables. 


Proof: 


Given x € X, Theorem 2.1 asserts that there exists a solution to Equation 
(2.3). Let r be the rank of the coefficient matrix associated with the equality 
system in Equation (2.3). Hence, r = min {(n + 1), (k + €)}. Furthermore, note 
that the set of (A, 44) satisfying Equation (2.3) forms a polyhedral set in cae 
and by Theorem 2.1, this set has an extreme point. Since there must be (k + ¢) 
linearly independent hyperplanes binding at any such extreme point and the 
equality system provides r such hyperplanes, we must have at least (k +  — r) 
additional 2;- and yw ;-variables equal to zero at an extreme point of Equation 


(2.3). Consequently, there exists a representation of x in Equation (2.3) in which 
at most r of the A;- and j;-variables are positive. This completes the proof. 
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Theorem 2.1 suggests a constructive algorithm for actually representing a 
given point xX e€ X in terms of no more than (# + 1) extreme points and n 


extreme directions, using a number of elementary computational operations such 
as additions, multiplications, and comparisons, which are bounded above by a 
polynomial in » of degree < 4. That is, it suggests a polynomial-time algorithm 


of complexity bounded by on’) to obtain such a representation (see Exercise 


2.56). Note that if X is bounded, then the algorithm uses no more than min {(n + 
1), k} of its extreme points in the representation. 


Example 2.7 
Consider the polyhedral set of Example 2.6 that is illustrated in Figure 2.16. Let 


us represent the point x=(3) €X as in Equation (2.3). Bound X by constructing 


the plane x + x) < M similar to Figure 2.19 to obtain the set Y. Proceeding 
as in the proof of Theorem 2.1, since no constraints of X are binding at X, pick 
d = (0, 1) arbitrarily. Thus 7, = max{y : (4,3) + (0,1) € X} =3, and hence 
we get y, = (4, 6). Since two linearly independent equations are binding at yj), 
we have that y, is an extreme point x, of Y. Next, we compute 7) = max 
{y : (4,3) + 7(0,-3) € X} = 1/3, and hence we get y> = (4, 2). Therefore, we 
have x = dy, + (1 - 5)¥> where 6 =(1/3)4/3) = 1/4. Hence, 


X = (1/4)x; + G/4)y>. 
Next, observe that one constraint x. = 2 is binding at Y>. Repeating the process 


at Y>, and using d = (1, 0) as a solution to the homogeneous system x7 = 0, we 
get 


(M - 6) 8/3 


— X, + ——————_-x 
2" (= 10/3) > (M1073) * 


where x3 = (4/3, 2) and x4 = (M- 2, 2). Substituting, we obtain 


1 M-6)  , 2 


xX =—x, + 3 X4. 
4 4(M — 10/3) (M — 10/3) 


Now, at x4 = (M — 2, 2), which is of the form of Equation (2.4), only one 
constraint (x. = 2) of X is binding. This gives via its homogeneous solution 
the extreme direction d, = (1,0) of X, and so we get the adjacent extreme point 
of x4 in X by searching along —d, as x3 = (4/3, 2). This gives x4 = X3 + 
@d,, where 8 =1(x4 —x3) =(M —10/3). Substituting for x4, we finally obtain 
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1 % 3(M - 6) 


a) Se ee reece rE YY ES 
4M ~10/3)°3 * (af ~ 10/3) * ai 


or 


which is of the required form of Equation (2.3). Note that this representation is 
not unique. Another representation for x is 


x= ts + ae + Ja 
Ng a 
Note from Figure 2.18 that n = 2, k = 3, and ¢@ = 2 in this problem. Hence, 
min{(n + 1), (k+ £)} = 3, and hence, both representations use the upper bound 
number of terms prescribed by Corollary 2.1. In this case, a lower bound on the 
number of terms required in any representation of x = (4, 3) also happens to be 
three. 


Some Insights into the Equality Constrained Polyhedron 


Consider the nonempty polyhedral set X = {x: Ax =b, x > 0} where Ais mxn 


and rank(A) = m. Note that by equivalently replacing each equality constraint by 
two inequalities, all the discussion in this section continues to hold true for this 
case as well. In particular, consider the extreme points of X. As before, these are 
points in X formed by the intersection of » linearly independent hyperplanes. 
However, from the equality constraints, we know that m linearly independent 
hyperplanes are always binding at any feasible solution. Hence, at any extreme 
point x of X, there must be some (n — m) additional hyperplanes binding from 


the nonnegativity constraints, say, x, = 0, which together with Ax = b pro- 


duce n linearly independent equations in m unknowns, thereby yielding X as the 
unique solution. Similarly, one can characterize edges and adjacent extreme 
points of X following Section 2.6, and note as before that extreme directions of 
X correspond to extreme points of 


D = {d: Ad = 0,1d = 1,d > 0}. 


Finally, in light of the remark concerning extreme points of X, the reader may 
find it instructive to read through the proof of Theorem 2.1, thinking of X as the 
equality constrained set. 


EXERCISES 
[2.1] Show that the vectors 
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0 
form a basis for R°. Supposing that a, is replaced by Hl indicate whether the 
1 


new set of vector still forms a basis of R. 


[2.2] Which of the following collection of vectors form a basis of R?, span R’, 
or neither? 


l 
BNW 
VS 


} fh 
| 


| 
af of 
A ol 
vb wt 
| 


[2.3] Let 


") 
3 |. 
5 


Are these vectors linearly independent? Do they span R ae 


[2.4] Let a,,a,...,a, form a basis for R”. Show that aj,a7,...,a, are linearly 
independent. Also show that k = n. 


[2.5] Does the following matrix have an inverse? If the answer is yes, find Al. 


1-4 4 0 
2 50 1 

A= . 
0 20 «1 
1 32 2 

[2.6] Find the inverse of the following triangular matrix: 

24-3 -4 
04 #3 #2 

A= : 
00 2 =5 
00 0 -1l 
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[2.7] Let A be a 3 x 5 matrix. Consider the matrix C obtained by weighting the 
columns of A with scalars 1, 3/2 , -2, 4, and —1, respectively, and adding them 


up. Write C as a matrix product. Similarly, let D be the weighted sum of the 
rows of A, using weights 2, —2, and 3/2, respectively. Write D as a matrix 
product. Likewise, let E be a matrix whose first row is the sum of the rows of A, 
whose second row is the third row of A, and whose third row is obtained by 
summing: thrice the first row of A, —5/2 times the second row of A, and twice 
the third row of A. Write E as a matrix product. 


[2.8] Let B be an invertible matrix. Show that B! is unique. 
[2.9] Suppose that a,,a4,...,a,, form a basis of R” and y = Aa; +Apan + + + 
A,a, with A; = 0. Prove that ay,...,8;1,Y, @j41,---.@, do not form a basis of R”, 


[2.10] Let 


where B is an m x m invertible matrix, I is a k x k identity matrix, 0 is an m x k 
zero matrix, and T is an arbitrary k x m matrix. Show that A has an inverse and 
that 


B10 
-TB! | 1} 


[2.11] Show that if A and B are n < n matrices that are both invertible, then 
(AB)! = BAT! 


[2.12] Let A be an 7 x 7 invertible matrix. Show that A’ has an inverse and that 
(Aly! = (Ay 

[2.13] Let B be an invertible matrix with nonnegative entries. Show that every 
row of B™! has a least one positive entry. 


[2.14] Let A = (a),89,...,a;,...,8,) be an invertible m x m matrix. Show that 


Ala j = &; where e; is a vector of zeros except for a 1 at position j. 


[2.15] Let A be an 2 x n matrix. Suppose that B is an ” x n matrix such that AB 
=I. Is it necessarily true that A has an inverse? Is it necessarily true that B = 


Al? 

[2.16] If the ith row of a square nonsingular matrix B is multiplied by a scalar 
A # 0, what changes would result in B!9 

[2.17] If the ith column of a square nonsingular matrix B is multiplied by a 


scalar 4 * 0, what changes would result in Bl? 
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[2.18] Find the rank of the following matrices: 


10 1 -1 

Gi) A=!2 3 4 #2}. 
10 5 -3 
—2 1 -3 

Gi) A=] 2 4 -2), 
43 1 

[2.19] Find the determinants of the following matrices. 

10 1 

a A=/2 2 -1). 
0 1 2 
10 2 1 
—3 1 -1 «1 

b. A= 
—2.2 -1 2 
1-2. 2-5 
22 #1 

c Az=]1 3 -2 
3 +2 -3 

[2.20] Solve the following system by Cramer’s Rule: 
2x) - XX = 6 
5x) + 2x, = 4. 
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[2.21] Demonstrate by enumeration that every basis matrix of the following 


system is triangular: 


xy + x3 + Xf = 4 
“XH oF Xp F XH OF 2 
- 2% - m = 3 
= 3 = —2, 


[2.22] Solve the following system of equations: 


X + 2x. + x3 = 1 
—Xy) + xX - 3 = 3 
2x) + 3x. + x3 = 4. 


Without resolving the system, what is the solution if the right-hand-side of the 


first equation is changed from 1 to 2? 


[2.23] Show that the determinant of a square triangular matrix is the product of 


the diagonal entries. 
[2.24] Find all basic solutions of the following system: 
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—Xy + 2X7 + xX, + 3x4 = 2X5 = 4 
xy 2X7 + 2X4 + xy = 2: 


[2.25] Determine whether the following system possesses: (a) no solution, (b) a 
unique solution, or (c) many (how many?) solutions. 


x} + 3x4 + 34 -— X= 1 
5x4 = 6x3 + Xf = 0 
x} = 2X7 + 4x, = 2. 


[2.26] Construct a general solution of the system Ax = b where A is anim xn 
matrix with rank m. What is the general solution of the following system? 
Xx -— 2x + »% = 2 
—Xy + 3x9 + 2x3 = 6. 


[2.27] Consider the system Ax = b where A = [a),4),...,a,,] is an m x n matrix 
of rank m. Let x be any solution of this system. Starting with x, construct a basic 
solution. (Hint: Suppose that x,,...,x, #0 and x,,1,....x, = 0. If p > m, repre- 


sent one of the columns a,, for / = 1, ..., p, as a linear combination of the 


J > 
remaining vectors. This results in a new solution having a smaller number of 
nonzero variables. Repeat the process.) 


[2.28] Which of the following sets are convex and which are not? 
a. {(X,x)) xp + x5 > 3}. 
b.  {(%15%25%3) 1X) + 2x) <1, 4 — 2x3 < 2}. 
c.  {(X15x2) xy — 3x7? = 0}. 
ds 4 axon ke) Fee x5 ap Di oy = 
xy| < 4}. 


e.  {(%,%)) 5 x = 3, 
f. {(%1,%9,%3) 1 3 = |x|, x, S$ 3}. 
[2.29] Which of the following functions are convex, concave, or neither? 

a. f(x) = x’. 

b £665) Se 4 eh Oe, 

c.f (%,x2) = maximum{ fi(x,x2), fo(%.x2)} where f((%1.x2) = 3x7 
+ x§ and fo(x,x2) = 2x7 - 5x9. 

df (x,x9) = minimum{ f(x,y), A(.x2)} where A) and AAC) 
are defined in Part (c). 

e. f(%1.%2,%3) = axe - ox = Ke + 2x1X_ — X9Xz + 2x, + 5x3. 


f.  f(%,x2) = af + xs — 2xjxX_ +X. 
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[2.30] Consider the set {(x,,x>) : —xy + X» $2, x1 + 2x $8, x4 2 0, xy > 0}. 
What is the minimum distance from (—3,4) to the set? What is the point in the 
set closest to (—3,4)? Repeat with the point (6, 4). 


[2.31] Let a, = (3). a, = ig) a3 = ia) a4 = (4) and as = (3) Tllus- 


trate geometrically the collection of all convex combinations of these five 
points. 


[2.32] Show that a hyperplane H = {x: px =k} and a half-space H* = {x: px > 
k} are convex sets. 

[2.33] Show that fis convex if and only if its epigraph ={(x,y): x € R”, y€ 
R, y = f(x)} is a convex set. Similarly, show that fis concave if and only if its 
hypograph = {(x,y) x € R", ye R', y < f(x} isa convex set. 

[2.34] Show that a differentiable function f is convex if and only if the 
following inequality holds for each fixed point xg in R”: f(x) = f(x9) + 


Vf (Xo (x - Xq) for all x <¢ R”, where V/(xq) is the gradient vector of fat xo 
given by 


Of (Xo) F(X) F(X) 
ax, 7° Oxy >” Ox 


n 


[2.35] Show that the set of feasible solutions to the following linear program 
forms a convex set: 


Minimize ex 
subject to Ax=b 
x> 0. 


[2.36] Consider the set XY = {(x1,X%2) 1% + Xp 2 2, XxX S 4, x,X 2 O}. 
Find a hyperplane H such that X and the point (1,—2) are on different sides of 
the hyperplane. Write the equation of the hyperplane. 

[2.37] If S is an open set, show that the problem 


Maximize cx 
subject to xeS 


where ¢ # 0 possesses no optimal point. (Note: S is open if for each Xg € S, 
there is an € > 0 such that |x Se Xo| < € implies that x € S.) 

[2.38] Show that C is a convex cone if and only if x and y € C imply that 
Ax + wy € C forall 2 > 0 and w = 0. 


[2.39] Show that if C is a convex cone, then C has at most one extreme point, 
namely, the origin. 
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[2.40] Find the extreme points of the region defined by the following 
inequalities: 


x| + 2x te x3 < § 
x, + x2 th 2x3 < 6 
XI; x2; x3 > 0. 


(Hint: Consider n = 3 intersecting defining hyperplanes at a time.) 


[2.41] Why does the following set have directions? Find all its extreme 
directions: 


—X} + 2X = 3 
2x _ 2x = x3 <. 2 
x3 = 1 

X15 x9, x3 2 0. 


(Hint: Enumerate the extreme points of its normalized set of directions. What 
else needs to be checked?) 


[2.42] Find all extreme points of the following polyhedral set: 
X= {(x1,%.X3) 1 XY — Xq +3 S Lx — 2xXQ S 4,%1,xX,x3 2 O}. 
Does X have any recession directions? Why? 


[2.43] Let X= {x : Ax < b} c R” and let x9 € X be such that fewer than ” 
linearly independent hyperplanes defining X are active at x9. Show that xo 
cannot be an extreme point of X. 

[2.44] Prove in detail that a polyhedral set XY is bounded if and only if it has no 
directions. 

[2.45] Consider the nonempty polyhedral set X = {x : Ax = b,x 2 0}. Show 
directly by definition that d is a direction of the set if and only ifd # 0, Ad =0, 
andd 2 0. Obtain analogous results if Ax = b is replaced by A)x = by, 
A>x 2 bp. 

[2.46] Given a nonempty polyhedron X, a face F of X is also defined as the 
intersection of a supporting hyperplane of X with X itself. (A hyperplane 
supports X if X lies completely in one half-space defined by the hyperplane and 


the intersection of the hyperplane with XY is nonempty.) Show the equivalence 
between this definition and the one given in Section 2.6. 


[2.47] You are given the following polyhedral set. Identify the faces, extreme 
points, extreme directions, and extreme rays of the set. 


x| i Xx + x3 < 10 
2x) -— X% + 2x, < 40 
3x, — 2x, + 3x, < 50 

x1, x2; x3 2 0. 
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[2.48] Let X ={(x,x%)) 2 x) — x2 $3, —x + 3xq $3, x 2 -3}. Find all 
extreme points of X and represent x = (0, 1) as a convex combination of the 
extreme points. 


[2.49] Answer the following questions and provide a brief explanation or 
illustration: 
a. Is it possible for X in Equation (2.1) to be empty but D in Equation 
(2.2) to be nonempty? 
b. Is there a relationship between redundancy and degeneracy of a 
polyhedral set? 
Does degeneracy imply redundancy in two dimensions? 
d. If the intersection of a finite number of half-spaces is nonempty, 
then this set has at least one extreme point. True or false? Explain. 
e. An unbounded n-dimensional polyhedral set can have at most n 
extreme directions. True or false? Explain. 
f. What is the maximum (actual) dimension of X = {x : Ax = b,x 


QO 


> 0}, where A is m x n of rank r, with r<m <n? 
[2.50] Find all extreme points and extreme directions of the following poly— 
hedral set: 
X = {(%,.%7,%3,%4) 2 Xp +X — 2x3 S11, 
2x —%3 + 2X4 < 2: X1 9% 5X3 5X4 > 0}. 

Represent x = (1, 1, 1, 2) as a convex combination of the extreme points of X 
plus a nonnegative combination of the extreme directions of X. 

[2.51] Show that an unbounded polyhedral set of the form {x : Ax = b, x>0} 


has at least one extreme direction. In particular, using the normalized direction 
set D in lieu of X in the proof of Theorem 2.1, show how you can start with any 
direction d in D and reduce it to an extreme direction. 


[2.52] Consider the polyhedral set X = {x : px = k} where p is a nonzero 


vector and k is a scalar. Show that X has neither extreme points nor extreme 
rays. How do you explain this in terms of the general Representation Theorem? 


[2.53] Let X= {x : Ax = b,x > 0} where A is an m x n matrix with rank m. 
Show that d is an extreme direction of X if and only if d is a positive multiple of 
the vector (-y',,0, 0,...51,0,...,0) where the | appears in position j, and where: 


Yj = Bla, <0 3 
= [B, N] where B is an m x m invertible matrix 
a; = acolumnofN. 


(Hint: Consider extreme points of the normalized direction set.) Illustrate by the 
following system: 
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oS 3x2 ia x3 = 0 
-3x + x2 + x4 = 3 
Xj; x2; x3; X4 = 0. 


[2.54] In the proof of Theorem 2.1, consider the direction d given by 
(xX, — X,))/@,. Show how you would algebraically obtain d, given the 


extreme point x, of X, v>k. 


[2.55] Consider the polyhedral set X = {x : Qx < q}, where Q is m x n. 
Mathematically characterize the statement that an entire line belongs to X. What 
does this imply about the rank of Q? Show that a nonempty polyhedron has 
extreme points if and only if it contains no lines. 

[2.56] Based on Theorem 2.1 and its proof, construct a polynomial-time 


algorithm of complexity bounded by O(n*) in order to obtain a representation 


of x e X in terms of no more than (n + 1) extreme points and n extreme 
directions of X. Show that the number of extreme points needed in the 
representation by this algorithm is no more than the bound provided by 
Corollary 2.1 when X is a bounded set. 


NOTES AND REFERENCES 

1. Sections 2.1 through 2.3 present a quick review of some relevant results 
of vector and matrix algebra. 

2; Sections 2.4 and 2.5 give some basic definitions and properties of convex 


sets, convex cones, and convex functions. For more details the reader 
may refer to Eggleston [1958], Mangasarian [1969], Bazaraa, Sherali, and 
Shetty [2006], and Rockafellar [1970]. 

3. Correspondence between bases and extreme points is established in the 
next chapter in Section 3.2, and an algebraic characterization of extreme 
directions is presented in Exercise 2.53. 

4. The representation theorem for polyhedral sets evolved from the work of 
Minkowski [1910] and Goldman and Tucker [1956]. The result is also 
true for (nonpolyhedral) convex sets that contain no lines. See 
Rockafellar [1970] and Bazaraa and Shetty [1976]. The geometric proof 
of Theorem 2.1 in Section 2.6 is taken from Sherali [1987b]. (Parts of the 
proof, parts of Example 2.7, and Figure 2.17 are reprinted from H. D. 
Sherali, “A Constructive Proof of the Representation Theorem for 
Polyhedral Sets Based on Fundamental Definitions,” American Journal 
of Mathematical and Management Sciences, Vol. 7 (1987), 253-270, 
Copyright © 1987 by the American Sciences Press, Inc., 20 Cross Road, 
Syracuse, New York 13224. Reprinted by permission.) For further 
geometric insights, see Akgul [1988], Grunbaum [1967], Murty [1983, 
1985], and Sommerville [1958]. 


THREE: THE SIMPLEX METHOD 


In this chapter we begin to discuss Dantzig’s simplex method, which was 
conceived in the summer of 1947 for solving linear programming problems. The 
first significant application of this method occurred soon after in the fall of 
1947. J. Laderman solved a diet-planning linear program with nine equality 
constraints and 27 nonnegative variables at the National Bureau of Standards. 
Using desk calculators, this problem took 120 man—days to solve and the 
worksheets were laboriously glued together and spread out like a “table cloth.” 
Today, using modern—age computer facilities and sophisticated implementations 
of the simplex method, linear programs having more than tens—of—thousands of 
constraints and variables are readily solvable. Although several variants of the 
simplex method have evolved and other new competing algorithms have been 
proposed (see Chapter 8), the simplex method remains a viable and popular tool 
for solving linear programming problems. It also provides further insights into 
the facial structure of polyhedral sets that define the underlying feasible region 
for linear programs. 

We begin our discussion of the simplex method by showing that if an 
optimal solution exists, then an optimal extreme point also exists. Extreme 
points are then characterized in terms of basic feasible solutions. We then 
describe the simplex method for improving these solutions until optimality is 
reached, or else until we conclude that the optimal value is unbounded. Both 
geometric and algebraic descriptions are provided. The well-known tableau 
format of the simplex method is also discussed. This is a key chapter, 
fundamental to the development of many other chapters in the book. 


3.1 EXTREME POINTS AND OPTIMALITY 


We observed from Figure 1.3 that when an optimal solution of a linear 
programming problem exists, an optimal extreme point also exists. This 
observation is always true, as will be shown shortly. 

Consider the following linear programming problem: 


Minimize cx 
subject to Ax=b 
x>0. 


Let X,, X2, .... X; be the extreme points of the constraint set, and let dy, do, ..., 
d,, be the extreme directions of the constraint set. Recall that any point x such 
that Ax = b and x > 0 can be represented as 


k £ 
j=l J=i 


where 
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k 

2 4; =] 

j=l 

Aj > 0, J=i, Jk 
Mj = 0, j=... 2 


Therefore, the linear programming problem can be conceptually 
transformed into a problem in the variables A), A5,...,A,, My» Lz 5--s Alp, Tesult- 
ing in the following equivalent linear program: 


k £ 
Minimize > (cx; Aj + ¥ (ed; )y; 
j=l j=l 


k 
subject to > Aj =| 


j=l 
Aj > 0, J=1,...k 
Hj > 0, J=A,.us & 


Assume that & > 1. Since the ;-variables can be made arbitrarily large, the 
minimum is —oo if ed ; <0 for some/ ¢€ {l,..., 2}. If ed | > 0 for all = 1,,..., 2, 
then the corresponding yz; can be chosen as zero. Now, in order to minimize 
Li (cx V, over Aj,Ag,...,A, satisfying A; 20 for j = 1,..., k, and 
pie Aj = 1, we simply find the minimum cx ;, Say CX), let Ay = ], and set 
all other A; 


To summarize, given feasibility, the optimal value of the linear problem 
is finite if and only if ed ; 2 0 for all extreme directions. Furthermore, if this is 


-variables equal to zero. 


the case, then we can find a minimizing point by selecting a solution having the 
minimum objective value among all extreme points. This shows that if an 
optimal solution exists, we must be able to find an optimal extreme point 


solution. Of course, if the minimum cx . value occurs at more than one index, 


then each corresponding extreme point is an optimal point and each convex 
combination of these points is an optimal solution (why?). In fact, the entire set 
of alternative optimal solutions is given by the set of convex combinations of 
such points plus a nonnegative linear combination of the extreme directions d ; 


J 
that satisfy ed ; = 0 (why?). 


Example 3.1 


Consider the region defined by the following constraints: 
—y + xX S 2 
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6 
0. 


—X} e520. x4 < 
2 


X15 x2 
Note that this region has three extreme points x,, X, and x3, and two 


extreme directions d, and d, (see Figure 3. 1). These are (without normalizing 
dy) 


Now, suppose that we are minimizing x; — 3 x. over the foregoing region. We 
see from Figure 3.la that the problem is unbounded and has value —oo. In this 


case, we have 
(1,-3) : 0 
Cc = oa Ss 
xX 0 
0 
= (1,-3 = -6 
cx, = ( iW 
2 
= (1,-3 = -10 
CX3 ( 1, 


1 
ed, = «34 =] 
2 
cd, = 0-9) 1] = -1, 


The preceding problem is equivalent to the following: 


Minimize 0A, -— 6A, - 104A, + wy - fy 
subjectto 24. + A + A 


IV Il 
ou 


A, A» Ag, Ms Mn 
x2) x2 

X3 x3 
Objective da Optimum? d 
decreases x 4 
indefinitel x x 
a a Objective c= (4,-) 

¢ = (1-3) decreases 


(a) (6) 


Figure 3.1. Extreme directions and optimality: (a) Unbounded optimal 
value. (b) Bounded optimal solution. 
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Since cd, = -1 <0 and “ can be made arbitrarily large without violating 
the foregoing constraints, the objective value can be made —oo by letting w = 
oo. Then ,4, can be chosen equal to zero. Any set of nonnegative 2, A, A3 
adding to | satisfies the foregoing constraints, for example, 4, =1, A) = 43 = 


0. This illustrates the necessary and sufficient condition for unboundedness for a 
feasible linear program, namely, ed < 0 for some extreme direction d. 
Now, consider the problem of minimizing 4x, — x over the same 


region. From Figure 3.1b the optimal solution is the extreme point x, = et In 


this case we have 


cd, = 


I 
o~ 
PP 

| 
—_ 
— 

| aa | 
aS) 
| 

I 


This problem is therefore equivalent to the following: 


Minimize 0A, -24 + 44, + 4m +7 
subjectto 2A +a + A = | 
A; Ay A3, bh, i 1S 0, 


Since the coefficients of 4, and jf in the objective function are 
positive, we let 44, =“ = 0. In order to minimize the expression 0A; — 2A, + 
4A, subject to A; + Ay +Az =I1 and A, A, Az 20, we let A = 1 and A, = 


A; = 0. This shows that the optimal solution is the extreme point x. = BI 


Minimizing ex corresponds to moving the plane cx = constant in the 
direction —c as far as possible. When c = (1, —-3) we can move the plane 
indefinitely while always intersecting the feasible region, and hence the optimal 
value is ~«. When ec = (4, -1) we cannot move the plane indefinitely in this 


fashion and we must stop at the point x; otherwise, we will “leave” the 
feasible region. 


3.2 BASIC FEASIBLE SOLUTIONS 


We have developed, in the previous section, a necessary and sufficient condition 
for an unbounded solution. We also showed that if an optimal solution exists, 
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then an optimal extreme point also exists. The notion of an extreme point is a 
geometric notion, and an algebraic characterization of extreme points is needed 
before they can be utilized from a computational point of view. 

In this section we introduce basic feasible solutions and show that they 
correspond to extreme points. This characterization will enable us to 
algebraically describe the simplex method. 


Definition (Basic Feasible Solutions) 


Consider the system Ax = b and x > 0, where A is an m x n matrix and b is an 
m-vector. Suppose that rank (A, b) = rank (A) = m. After possibly rearranging 
the columns of A, let A = [B, N] where B is an m x m invertible matrix and N is 


an m X (n ~ m) matrix. The solution x = [28 to the equations Ax = b, where 
N 


xz = B'b 
and 
XN =0 


is called a basic solution of the system. If xp > 0, then x is called a basic 
feasible solution of the system. Here B is called the basic matrix (or simply the 
basis) and N is called the nonbasic matrix. The components of xg are called 


basic variables (or dependent variables) and the components of xj, are called 
nonbasic variables (or independent variables). If xp > 0, then x is called a 
nondegenerate basic feasible solution, and if at least one component of xp is 


zero, then x is called a degenerate basic feasible solution. 
The notion of a basic feasible solution is illustrated by the following two 
examples. 


Example 3.2 
(Basic Feasible Solutions) 


Consider the polyhedral set defined by the following inequalities (and illustrated 
in Figure 3.2): 


6 
3 
0 


bay 
N 
IV IA IA 


xy, x2 


By introducing the slack variables x, and x4, the problem is put in the 
following standard format: 
Hy + X% + % 
x2 + X4 
> *2> %3> X4 


Vit lt 


Own 
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Figure 3.2. Basic feasible solutions. 


11 1 0 
010 1 


foregoing definition, basic feasible solutions correspond to finding a 2 x 2 basis 


Note that the constraint matrix A = [a),a7,83,a4] = | From the 


B with nonnegative B’'b. The following are the possible ways of extracting B 
out of A. 


l. B = [a1.a2] =| 1 


3. B= [aaa] =|] a| 

o=[2]-e=[1 il]-[3} «» -[2]-[8} 
4. B=[a.a]=[} 4] 

vo =[f3]-B=[1 TES]-[ 4} = -[8]-[o) 
5. B=lasa)=[4 9] 


xe =[B]-B'-[o “I3]- 5} -L2]-[2} 


Note that the points corresponding to 1, 2, 3, and 5 are basic feasible solutions. 
The point obtained in 4 is a basic solution, but it is not feasible because it violates 
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the nonnegativity restrictions. In other words, we have four basic feasible 
solutions, namely: 


These points belong to R4, since after introducing the slack variables we have 


four variables. These basic feasible solutions, projected in R*— that is, in the 
(x,X2) space—give rise to the following four points: 


5} [ob [3b ae [o} 


These four points are illustrated in Figure 3.2. Note that these points are 
precisely the extreme points of the feasible region. 

In this example, the possible number of basic feasible solutions is 
bounded by the number of ways of extracting two columns out of four columns 
to form the basis. Therefore the number of basic feasible solutions is less than or 
equal to 


Out of these six possibilities, one point violates the nonnegativity of Bb. 
Furthermore, a; and a3 could not have been used to form a basis, since a) = a3 


= | are linearly dependent, and hence the matrix E 0 does not qualify as 


a basis. This leaves four basic feasible solutions. In general, the number of basic 
feasible solutions is less than or equal to 


(7) = oa 


There is another intuitive way of viewing basic solutions and basic feasible 
solutions. (This is made more precise in the sequel.) Each constraint, including 
the nonnegativity constraints, can be associated uniquely with a certain variable. 
Thus x, = 0 can be associated with the variable x,, and the line x, = 0 defines 
the boundary of the half-space corresponding to x, = 0. Also, x; + x2 S$ 6 can 
be associated with the variable x3, and x3 = 0 defines the boundary of the half- 
space corresponding to x + x2 <6. Figure 3.3 portrays graphically the 


boundaries of the various half—spaces defined by the constraints. Now, basic 
solutions correspond to the intersection of two lines in this graph. The lines 
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Figure 3.3. Associating basic solutions with nonbasic variables. 


correspond to the nonbasic variables. In the graph there are five such 
intersections corresponding to five basic solutions. Note that there is no 
intersection of the lines x, = 0 and x4 = 0, and thus no basic solution 


corresponds to these two variables being nonbasic. Once the feasible region is 
identified, we can distinguish the basic solutions from those that are also basic 
feasible solutions. 


Example 3.3 


(Degenerate Basic Feasible Solutions) 


Consider the following system of inequalities: 


xy OF x2 < 6 

x2 < 3 
xX + 2x < 9 
xX}, xy 2 0. 


This system is illustrated in Figure 3.4. Note that the feasible region is precisely 
the region of Example 3.2, since the third restriction x, + 2x. <9 is 


“redundant.” After adding the slack variables x3, x4, and x5, we get 


Xy + xX + & = 6 

x2 + X4 = 3 

xX} + 2X, + x5 = 9 

xX, x2, x3, X4, Xs 2 0. 

Note that 

1110 0] 

A= [ay, a2, @3, a4, as] =/0 10 10 
1200 1] 


Let us consider the basic feasible solution with B = [a;, a7, a3]: 


x 1 1 1) [6] fo -2 176) f3 
xg =|}x,/=]0 1 0] [3/=/0 1 off3|=]31, 
xa} [1 2 of} [of fa 1 1/9] [0 
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Figure 3.4. Degenerate basic feasible solution. 


EL 


Note that this basic feasible solution is degenerate since the basic variable 
x3 = 0. Now, consider the basic feasible solution with B = [a,,a>,a4]: 


x 1 1 0] [6 2 0 -1][6] [3 
Xp ={%.}/=/0 1 1) |3}=]-1 0 113] =] 3), 
X4 1 2 0} {9 1 1 -1]/9 0 
-|%3]-|0 
Tee [2 : 3 
Note that this basis gives rise to the same basic feasible solution obtained by 
B = [a),a,43]. We can also check that the basic feasible solution with basis 


B = [a;,a2,a5] is given by 


2 : %3 0 
Xp =|] =] 3), xy =| ee 

Xe 0 *4 0 
Note that all three of the foregoing bases represent the single extreme point or 
basic feasible solution (%,,x2,x3,%4,x5) = (3, 3, 0, 0, 0). This basic feasible 
solution is degenerate since each associated basis involves a basic variable at 
level zero. The remaining extreme points of Figure 3.4 correspond to 
nondegenerate basic feasible solutions (why?). The reader should also note that 


degeneracy is not always simply the result of redundant constraints (see Figure 
2.14, for example). 


Correspondence Between Basic Feasible Solutions and Extreme Points 


We shall now show that the collection of basic feasible solutions and the 
collection of extreme points are equivalent. In other words, a point is a basic 
feasible solution if and only if it is an extreme point. Since a linear 
programming problem having a finite optimal value has an optimal solution at 
an extreme point, an optimal basic feasible solution can always be found for 
such a problem. 
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Consider the following problem: 
Minimize ex 
subjectto Ax=b 
x>0, 
where A is an m x n matrix with rank m. Let x be an extreme point of the 
feasible region. We shall show that x is also a basic feasible solution of the 
system Ax =b,x>0. 

By the definition given in Section 2.6, there are some vn linearly 
independent defining hyperplanes binding at x. Since Ax = b provides m 
linearly independent binding hyperplanes, there must be some p = n -— m 
additional binding defining hyperplanes from the nonnegativity constraints that 
together with Ax = b provide n linearly independent defining hyperplanes 
binding at x. Denoting these p additional hyperplanes by x,y = 0, we therefore 


conclude that the system Ax = b, xq = 0 has x as the unique solution. Now, let 
N represent the columns of the variables x, in A, and let B be the remaining 
columns of A with xp as the associated variables. Since Ax = b can be written 


as Bxz + Nx, =b, this means that B is m x m and invertible, and moreover, Xp 


= Bb > 0, since x = (xp, Xj)) is a feasible solution. Therefore, x is a basic 


feasible solution. 
Conversely, suppose that x is a basic feasible solution of the system Ax = 
b, x = 0. We want to show that x is an extreme point. By definition, x = (xg, 


Xj) where correspondingly A = (B, N) such that xp = Bb >0and x wn =O. 
But this means that the » hyperplanes Ax = b, xy = 0 are binding at x and are 


moreover linearly independent, since B! exists. Hence, by the definition in 
Section 2.6, x is an extreme point. Therefore, we have shown that every basic 
feasible solution is an extreme point and vice versa. Exercise 3.15 asks the 
reader to construct an alternative proof for this characterization. 

Every basic feasible solution is equivalent to an extreme point. However, 
there may exist more than one basis corresponding to the same basic feasible 
solution or extreme point. A case of this type can occur in the presence of 
degeneracy, as illustrated in Example 3.3. Referring to the preceding proof, this 
case corresponds to that of an extreme point at which some r > p =n — m defining 
hyperplanes from x > 0 are binding. Hence, for any associated basis, (r — p) of the 
Xp -variables are also zero. Consequently, the number of positive variables is g = 


m — (r — p) <m. In this case, each possible choice of a basis B that includes the 
columns of these qg positive variables represents this point (why?). 

Clearly, if there exists more than one basis representing an extreme point, 
then this extreme point is degenerate. However, the converse is not necessarily 
true. Consider the following example of a polyhedral set: 


Xx + X% + % = 1 
“y+ X + % = 1 
xX, x2, x3, 2 0. 
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Consider the solution x = (0, 1, 0). Observe that this is an extreme point or a 
basic feasible solution with a corresponding basis having x, and x» as basic 
variables. Moreover, this is a degenerate extreme point. There are four defining 
hyperplanes binding at x. Moreover, there are three ways of choosing three 
linearly independent hyperplanes from this set that yield x as the (unique) 
solution (how?). However, the basis associated with x is unique. 

In this example, the constraints Ax = b imply that x, = 0. However, after 


fixing x, = 0, we no longer have a full rank system. If this were not the case, then 
for any degenerate extreme point, there would be more than one corresponding 
basis. To see this, consider the system Ax = b, x > 0, where A is m x n of rank m. 
Let x be a degenerate extreme point solution, and let xg and xj bea set of basic 
and nonbasic variables, respectively, corresponding to a particular basis. As usual, 
let B and N be the columns of A associated with the variables xg and xy, 
respectively. Then, the given system is equivalent to 


B'Nx, +x, = B b=b 


Xn > XB >0. 


Consider a degenerate basic variable xB. (with b, = 0), which is such that Ax 
= b does not necessarily imply that xB = 0. Given that such a variable exists, 
we will construct another basis representing this point. Let x, be some 
component of x, that has a nonzero coefficient 0, in the row corresponding to 
xp . Note that x, exists. Then consider a new choice of (n — m) nonbasic 
variables given by xg and xy_,, where xy_, represents the components of 


Xy Other than x, . Putting xB. = O and xy_, = 0 above uniquely gives x, = 


b,/6, = 0 from row r, and so xB = b; is obtained as before from the other 


rows. Hence, this corresponds to an alternative basis that represents the same 
extreme point. Finally, note that if no degenerate basic variable xB. of this type 


exists, then there is only one basis that represents this extreme point. 


Existence of Basic Feasible Solutions 


Since a nonempty set X = {x: Ax = b, x > 0} has extreme points (from Theorem 
2.1), we know that the system Ax = b, x > 0 has at least one basic feasible 
solution. In fact, starting from any feasible point x e X, the proof of Theorem 
2.1 shows how to constructively obtain an extreme point, and hence a basic 
feasible solution of X. Assuming without loss of generality that rank (A) = m, 
we can examine the binding nonnegativity constraints at this extreme point, and 
select p = (n — m) of them to correspond to the variables xj, such that the 


system Ax = b, xy = 0 has rank 7 (how?). Calling the remaining m variables 


Xp_ as basic variables, we obtain a corresponding basis. 
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Summary of Results 


Let us summarize some of the important facts about the following linear 
programming problem, where A is an m X n matrix with rank m. 


Minimize cx 
subject to Ax=b 
x>0. 


Theorem 3.1 


The collection of extreme points corresponds to the collection of basic feasible 
solutions, and both are nonempty provided that the feasible region is not empty. 


Theorem 3.2 


Assume that the feasible region is nonempty. Then a finite optimal solution 
exists if and only if ed ; > 0 for/ = 1,..., €, where dj,...,d, are the extreme 


directions of the feasible region. Otherwise, the optimal solution value is 
unbounded. 


Theorem 3.3 


If an optimal solution exists, then an optimal extreme point (or equivalently an 
optimal basic feasible solution) exists. 


Theorem 3.4 


For every extreme point (basic feasible solution) there is a corresponding basis 
(not necessarily unique), and, conversely, for every basis there is a correspond- 
ing (unique) extreme point. Moreover, if an extreme point has more than one 
basis representing it, then it is degenerate. Conversely, a degenerate extreme 
point has more than one basis representing it if and only if the system Ax = b 
itself does not imply that the degenerate basic variables corresponding to an 
associated basis are identically zero. 

Now, since the extreme points may be enumerated by algebraically 
enumerating all basic feasible solutions, which are bounded by (7) one may 
think of simply listing all basic feasible solutions and picking the one having the 
minimal objective value. This is unsatisfactory, however, for a number of 
reasons. First, the number of basic feasible solutions is bounded by fal which 
is large, even for moderate values of m and n. Second, this simple approach does 
not tell us if the problem has an unbounded optimal value, which may occur if 
the feasible region is unbounded. Last, if the feasible region is empty and if we 
apply the foregoing “simple-minded procedure,’ we shall discover that the 


feasible region is empty, only after all possible ways of extracting m columns 
out of n columns of the matrix A fail to produce a basic feasible solution, either 


on the grounds that B does not have an inverse, or else that Bb x 0. 


The simplex method is a clever procedure that moves from one extreme 
point to another extreme point having a better (at least not worse) objective 
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value. It also discovers whether the feasible region is empty and whether the 
optimal objective value is unbounded. In practice, the method only enumerates a 
small portion of the extreme points of the feasible region. The key to this 
method is revealed next. 


3.3 KEY TO THE SIMPLEX METHOD 


The key to the simplex method lies in recognizing the optimality of a given 
extreme point solution based on local considerations without having to 
(globally) enumerate all extreme points or basic feasible solutions. 
Consider the following linear programming problem: 
LP: Minimize ex 
subjectto Ax=b 
x>0, 


where A is an m X n matrix with rank m. Suppose that we have a basic feasible 


-1 
solution fe 0 >) whose objective value Zp is given by 


Zz) = se = (en.ew)( BQ? | = ¢pB"'b. (3.1) 


Now, let xp and x, denote the set of basic and nonbasic variables for the 
given basis. Then feasibility requires that xp > 0, x, > 0, and that b = Ax = 


Bxp + Nxj,. Multiplying the last equation by B™! and rearranging the terms, 
we get 


Xp = B'b - B'Nx,y 


= B'b - > B'a;x, 
jed 

= b = v3 (y ;)x;> Say, (3.2) 
jet 


where J is the current set of the indices of the nonbasic variables. Noting 
Equations (3.1) and (3.2) and letting z denote the objective function value, we 
get 


Zz = x 
= CpXp + Cy Xn 
= ¢p/B'b- Y Bla x; |+ D e;x, (3.3) 
Jed fed 
= 29> Xj ex; 


where z; = ¢ pB la ; for each nonbasic variable. 
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Using the foregoing transformations, the linear programming problem LP 
may be rewritten as: 


Minimize =z =z — > (z; ~ ¢;)x; 


jeJ 
subject to Dd (yx; + Xp = b (3.4) 
jeJ 
xj >0, jedJ, and Xp 20. 


Without loss of generality, let us assume that no row in Equation (3.4) has all 
zeros in the columns of the nonbasic variables x as j € J. Otherwise, the basic 


variable in such a row is known in value, and this row can be deleted from the 
problem. Now, observe that the variables xp simply play the role of slack 
variables in Equation (3.4). Hence, we can equivalently write LP in the nonbasic 
variable space, that is, in terms of the nonbasic variables, as follows: 
Minimize z= 2 — D(z; — c,)x; 
jeJ 
subjectto > (y,)x; <b (3.5) 
jet 
Xj >0, je J. 
Note that the number of nonbasic variables is p = (n — m), and so we have 


represented LP in some p—dimensional space. This is to be expected since there 
are p independent variables or p degrees of freedom in our constraint system. 


The values (c; — z; ) are referred to as reduced cost coefficients since they are 
the coefficients of the (nonbasic) variables in this reduced space. The 
representation (3.4), in which the objective function z and the basic variables 
Xp have been solved for in terms of the nonbasic variables, is referred to as a 


representation of the basic solution in canonical form. The key result now 
simply says the following: 


If (z; - c;) <0 for all j € J, then the current 


basic feasible solution is optimal. (3.6) 


This should be clear by noting that since z 7 ae 0 for all 7 € J, we have z 


> zo for any feasible solution; but for the current (basic) feasible solution, we 


know that z= zo, since x; =0 forall j € J. 


3.4 GEOMETRIC MOTIVATION OF THE SIMPLEX METHOD 


It is instructive to examine the foregoing operations geometrically. Observe that 
in the nonbasic variable space representation, the feasible region of LP is 
defined in terms of intersecting half—spaces: m associated with the inequality 
constraints in Equation (3.5), and p = n — m associated with the nonnegativity 
constraints. Associated with each of these half—spaces, there is a certain defining 
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Optimal solution 


x} 


Figure 3.5. Geometric motivation. 


variable that takes on a value of zero on the corresponding defining hyperplane, 
and takes on nonnegative values over the corresponding half-space. For the 
inequalities in Equation (3.5), these defining variables are the basic variables 


Xg, and for the nonnegativity constraints, these defining variables are the x;, j 


€ J, themselves. 
Figure 3.5 illustrates a situation in p = 2 dimensions, where n = 6, m = 4, J 
= {1, 2}, and where xp = {x3,x4,%5,X6}. The defining variables associated with 


the » = 6 defining hyperplanes are shown against the corresponding hyperplanes 
in this figure. The extreme points or vertices of the feasible region are labeled as 


V1 5--..¥5, and ¢ is the reduced cost vector. Note that the feasible region is defined 
by the restrictions x oe 0, 7 = 1,..., 2, in this space. The current basic feasible 


solution corresponds to the vertex v,, that is, the origin. Notice also that every 


other extreme point is also defined by the intersection of some p = 2 linearly 
independent hyperplanes, with the corresponding p defining variables being the 
nonbasic variables. Hence, for the vertex v defined by x, =0 and x, = 0, the 


nonbasic variables are x, and x3, and the basic variables are x7, x4, x5, and X¢. 
On the other hand, note that the degenerate vertex v3 has three defining hyper- 


planes passing through it, any two of which uniquely define this extreme point as 
their intersection. Hence, there are three bases representing v3. (Enumerate them.) 


Now, consider the origin v, and examine the p defining hyperplanes 


associated with the corresponding nonbasic variables. Holding (p ~ 1) of these 
hyperplanes binding and moving in a direction feasible to the remaining 
hyperplane takes us along a one—dimensional ray with vertex at v,. There are p 


such rays. Hence, at v,, since (p — 1) = 1, holding x, = 0 and increasing x 
takes us along the x, axis, with the objective function changing at a rate of 
0z/ Ox, = -(z,-c¢,) = G <0. Similarly, holding x, =0 and increasing x, takes 
us along the x» axis, with the objective function changing at a rate of 6z/@x. = 


— (22 ~-¢)) = ¢) <0. Hence either ray describes an attractive direction of motion. 
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Suppose that we choose the latter direction of motion. Because this is a feasible 
direction, it takes us along an edge of the feasible region (why?). Naturally, we 


would like to move as far as possible along this edge since 0z/@xy = C) is a 
negative constant. However, our motion is blocked by the hyperplane x; = 0, 


since x3 has been driven to zero and moving any further would drive x3 


negative. (Of course, if no such blocking hyperplane existed, then the optimal 
solution value would have been unbounded.) Furthermore, since (p — 1) linearly 
independent hyperplanes were binding all along and we were blocked by a new 
hyperplane, we now have p linearly independent hyperplanes binding, and so we 
are at another extreme point of the feasible region. In fact, this is an adjacent 


extreme point with respect to the previous extreme point (why?). At v2 the 
nonbasic variables are x, and x3, and the remaining variables are basic. We 


have now completed one step known as an iferation or pivot of the simplex 
method. In this step, the variable x, is called the entering variable since it 


entered the set of basic variables, and the variable x3 is called the blocking 
variable, or the leaving variable, since it blocked our motion or left the set of 
basic variables with respect to v;. The previous and the new basis differ in only 
one basic variable, and therefore, in only one nonbasic variable. Such bases are 
called adjacent bases. 

Repeating this process at vy, we would of course not like to enter x3 
into the basis, since it will only take us back along the reverse of the previous 
direction. However, holding x; = 0 and increasing x, takes us along an 
improving edge (why?). Proceeding in this direction as shown in Figure 3.5, we 
notice that more than one hyperplane blocks our motion. Suppose that we 
arbitrarily choose one of these, namely, x, = 0 as the blocking hyperplane. 


Hence, xg is the leaving variable, and for current basis representing v3, we 
have that x; and x4 are the nonbasic variables. Now, if we hold x, = 0 and 
move in a direction along which x3 increases, the objective function value 


would decrease because this direction makes an acute angle with —e. However, 
this direction is not feasible. We are blocked by the hyperplane x; = 0 even 


before we begin to move! That is, x5 leaves the basis giving x4 and x5 as our 
new nonbasic variables, while we are still at the same vertex v3. Such a step in 


which one basis is exchanged for an adjacent basis, both representing the same 
extreme point solution, is called a degenerate iteration or a degenerate pivot. 
While such a pivot is undesirable, it may not always be avoidable, unlike as in 
Figure 3.5 (how?). Certainly, we would not like to be stuck in an infinite loop at 
an extreme point while performing a sequence of degenerate pivots that takes us 
in a closed loop of bases, all representing the same extreme point, with the 
objective function therefore remaining a constant. Such a phenomenon is called 
cycling and, as shown in Chapter 4, it can indeed occur. Special precautions 
need to be taken for its prevention. 
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® 
p= ] p= 2 p= 3 
Figure 3.6. Examples of simplices in R’, p = 1, 2, 3. 


With x4 and xs as the nonbasic variables at v3, holding x5 = 0 and increasing 
x4 takes us along an improving, feasible edge direction and therefore results in 


a nondegenerate pivot. The blocking or leaving variable that gets driven to zero 
is xg, and the new nonbasic variables are x5 and xg. Observe that corresponding 


to this basis, none of the p rays, defined by holding some (p — 1) nonbasic 
variables equal to zero and increasing the remaining nonbasic variable, lead to 
an improvement in the objective function, and so our “key result” declares v4 


to be optimal. In fact, v4 is optimal in the polyhedral cone formed by the half— 


spaces having the current nonbasic variables as defining variables, which indeed 
contains the feasible region (why?). A path followed by the simplex algorithm 
along edges of the polyhedron (from v; to vz to v3 to v4 in the previous 


example) is known as a simplex path. 

The reader may note that when we implement the simplex method 
algebraically, we will be representing the linear program in the nonbasic 
variable space at every iteration, just as we did at the extreme point v,. This 


will conveniently give us the rate of change in objective function value along 
each of the p incident rays directly from the corresponding reduced cost vector, 
and so at each iteration, we will be inquiring whether the current origin is 
optimal or whether we ought to consider moving along one of the current “axis” 
directions. 

Finally, let us point out the origin of the term “simplex” in this algorithm 
(which is not an antonym of “complex”!). A simplex in p dimensions is the 


convex hull of a set of (p + 1) noncoplanar points in R”, that is, points not all 


lying on the same hyperplane in R”. Hence, for p = | this is a line segment; for 


p = 2 itis a triangle, and for p = 3 it is a tetrahedron, as shown in Figure 3.6. In 
particular, given a basis corresponding to an extreme point, examine for 
convenience the feasible region represented in the space of the nonbasic 
variables. Note that the convex hull of the current vertex taken as the origin, and 
some p other points, with one point chosen along each axis, defines a simplex in 


R? (why?). In fact, by examining the axis directions, the simplex method is 
verifying whether the origin is optimal for this simplex or whether any of the 
other vertices of the simplex have a better objective value. Consequently, the 
simplex method proceeds by examining one such simplex after another, each 
time inquiring whether the current vertex of the present simplex is optimal for 
this simplex or not. 
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3.5 ALGEBRA OF THE SIMPLEX METHOD 


We will now translate the geometric process described in the foregoing section 
into algebra. Toward this end, consider the representation of the linear program 
LP in the nonbasic variable space, written in equality form as in Equation (3.4). 


If (z;-c,;) <0 forall jeJ, then x; =0 for jeJ/ and xg = b is optimal 


for LP as noted in Equation (3.6). Otherwise, while holding (p — 1) nonbasic 
variables fixed at zero, the simplex method considers increasing the remaining 
variable, say, x, . Naturally, we would like z, —c, to be positive, and perhaps 


the most positive of all the z;-c;, j ¢J. Now, fixing x; =O for jeJ — {k}, 
we obtain from Equation (3.4) that 


Z= 29 —(ZR -CK) XE (3.7) 
and 
78 b Vik | 
*By | | by | | Vax 
ioe fee ca (3.8) 
XB b, Mrk 
xB, bin Ymk J 


If yi, <0, then xg, increases as x, increases, and so xg, continues to be non— 
negative. If yz > 0, then xg will decrease as x, increases. In order to satisfy 
nonnegativity, x; is increased until the first point at which some basic variable 


XB, drops to zero. Examining Equation (3.8), it is then clear that the first basic 


variable dropping to zero corresponds to the minimum of b; / yx for positive 


¥ix- More precisely, we can increase x, until 


b, b; 
Xx, =—> = minimum, —: yj, > OF. (3.9) 
Vrk Isism | Vix 


In the absence of degeneracy, b, > 0, and hence x, = b,/y,4 > 0. From 
Equation (3.7) and the fact that z, —c, > 0, it then follows that z< zg and the 


objective function strictly improves. As x, increases from level 0 to b,. Vip» @ 


new feasible solution is obtained. Substituting x; = b,/y,, in Equation (3.8) 
gives the following point: 
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Xp =}, -2£3,, i= 1,...,m, 
Vek 
b, 
x=, (3.10) 
Vrk 
and all other x ; variables are zero. 


J 
From Equation (3.10), x B= 0 and hence, at most m variables are positive. 
The corresponding columns in A are AB» ABs AB > Aes AB AB - Note 
that these columns are linearly independent since y,, # 0. (Recall that if ag,,..., 
AR sag are linearly independent, and if a, replaces a B, then the new columns 


are linearly independent if and only if y,, #0; see Section 2.1.) Therefore, the point 


given by Equation (3.10) is a basic feasible solution. 

To summarize, we have algebraically described an iteration, that is, the 
process of transforming from one basis to an adjacent basis. This is done by 
increasing the value of a nonbasic variable x, with positive z,—c, and 


adjusting the current basic variables. In the process, the variable xB drops to 
zero. The variable x, hence enters the basis and xg_ leaves the basis. In the 


absence of degeneracy the objective function value strictly decreases, and hence 
the basic feasible solutions generated are distinct. Because there exists only a 
finite number of basic feasible solutions, the procedure would terminate in a 
finite number of steps. 


Example 3.4 


Minimize x, + x2 


subjectto x, + 2x, < 4 
xX < | 
Xp xX 2 0. 


Introduce the slack variables x3 and x4 to put the problem in a standard form. 
This leads to the following constraint matrix A: 


1210 
A= [aj.az, a3, 94) =| 10 | 


Consider the basic feasible solution corresponding to B = [a), a>]. In other 
words, x, and x» are the basic variables, while x; and x4 are the nonbasic 


variables. The representation of the problem in this nonbasic variable space as in 
Equation (3.4) with J = {3, 4} may be obtained as follows. First, compute: 


Bt =|) 7’ -[3 “ak ,B = (1, | y]-a-n. 
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Hence, 

pme(s AOL) 

rowacls IL 
and 

somos IE 
Also, 


zy = ¢gB'b = 0.-n(7) =3, 
23 —C; =¢gB 'a;—c, =,-n]4]-0 =1, 


Z4—-C4q4 =c,B ‘a, -—C4 = (.-n]9]-0=-1. 


Hence, the required representation of the problem is: 


Minimize 3 - x, + Xx 


subject to x3 -— 2xy + = 22 
X4 + 2 = ] 
X15 X95 X3, X4 2 0. 


The feasible region of this problem is shown in Figure 3.7 in both the original 
(%1,X2) space as well as in the current (x3,x4) space. Since z3;—c3 > 0, then 


the objective improves by increasing x3. The modified solution is given by 


Xz= Bb -B'a3x, 


xX }_ 2 ~ ] x 

x2 “Td 0 3 
The maximum value of x3 is 2 (any larger value of x3 will force x, to be 
negative). Therefore, the new basic feasible solution is 


(%1,%2,X3,X4) = (0,1,2,0). 


Here, x3 enters the basis and x, leaves the basis. Note that the new point has an 
objective value equal to 1, which is an improvement over the previous objective 
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Figure 3.7. Improving a basic feasible solution. 


value of 3. The improvement is precisely (23 —c3 )x3 = 2. The reader is asked 
to continue the improvement process starting from the new point. 


Interpretation of Entering and Leaving the Basis 


We now look more closely at the process of entering and leaving the basis and 
the interpretation for these operations. 


INTERPRETATION OF z; -c; 


The criterion z, —c, > 0 for a nonbasic variable x, to enter the basis will be 
used over and over again throughout the text. It will be helpful at this stage to 
review the definition of z, and make a few comments on the meaning of the 


entry criterion z, —c, > 0. Recall that z= cpb—(z, —cy)x,, where 


m 
-1 
zy =CpB ay =Cayg = DCB, Vik (3.11) 
i=l 
and cp, is the cost of the ith basic variable. Note that if x, is raised from zero 


level, while the other nonbasic variables are kept at zero level, then the basic 
variables xg, Xg,,-» Xg,, must be modified according to Equation (3.8). In 


other words, if x, is increased by one unit, then xg, xg,, ...,and xg will be 
decreased respectively by yz, 2x5 -+-5 Vk units (if yy <0, then xg, will be 


increased). The saving (a negative saving means more cost) that results from the 
modification of the basic variables, as a result of increasing x, by one unit, is 


therefore })"", Cp, ¥ig» Which is z, (see Equation 3.11). However, the cost of 


increasing x, itself by one unit is cy. Hence, z, — cy is the saving minus the 
cost of increasing x, by one unit. Naturally, if z, — cg is positive, it will be to 
our advantage to increase x,. For each unit of x,, the cost will be reduced by 
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an amount z, — c,, and hence it will be to our advantage to increase x, as 


much as possible. On the other hand, if z, — c, <0, then by increasing x,, the 
net saving is negative, and this action will result in a larger cost. So this action is 
prohibited. Finally, if z; — c,= 0, then increasing x; will lead to a different 


solution having the same cost. So, whether x; is kept at zero level, or increased, 


no change in cost takes place. 
Now, suppose that x; is a basic variable. In particular, suppose that x, 1s 


the ‘th basic variable, that is, x, = xg, cy = Cg, and a, = ag. Recall that z; 
= cpB ‘a; = cgB ‘az. But Bla, is a vector of zeros except for one at the 
ith position (see Exercise 2.14). Therefore, z, = cg,, and hence, z, —cy = cg — 
CB, =0. 

Leaving the Basis and the Blocking Variable 


Suppose that we decided to increase a nonbasic variable x, with a positive z, — 
c,. From Equation (3.7), the larger the value of x;, the smaller is the objective 
z. As x; is increased, the basic variables are modified according to Equation 
(3.8). If the vector y,; has any positive component(s), then the corresponding 
basic variable(s) are decreased as x, is increased. Therefore, the nonbasic 
variable x, cannot be indefinitely increased, because otherwise, the nonnegativ- 
ity of the basic variables will be violated. Recall that a basic variable xg that 


first drops to zero is called a blocking variable because it blocks the further 
increase of x,. Thus, x, enters the basis and xg leaves the basis. 


Example 3.5 
Minimize 2x, — xX 
subject to —x; + X, S< 2 
2x + X S 6 
xX xX, 2 0. 


Introduce the slack variables x; and x4. This leads to the following constraints: 


—% + X) + % ='-2 
2x, + Xq + x4 = 6 
X> x2; x3; x4 > 0. 


Consider the basic feasible solution with basis B =[aj,a> ] -| ‘ i and 


LB 8 


The Simplex Method 113 


x, =B 'b-B'Nxy 
2-2/8 SEHR BE TE] 


4/3 1/3 /3 
(1 BB) La) oo 
Currently, x; = x4 =0, x, =4/3, and x, = 10/3. Note that 

Z4 7 C4 =epB!ay-cy =(2,-1 | a valk |-o-y3 >0. 


Hence, the objective improves by holding x3 nonbasic and introducing x, in the 
basis. Then x3 is kept at zero level, x4 is increased, and x; and x2 are modified 
according to Equation (3.12). We see that x, can be increased to 4, at which 
instant x, drops to zero. Any further increase of x4 results in violating the 
nonnegativity of x,, and so, x, is the blocking variable. With x, =4 and x; = 
0, the modified values of x, and xz are 0 and 2, respectively. The new basic 
feasible solution is 


(x, X25 X35 x4) = (0, 2, 0, 4). 


Note that a, replaces aj; that is, x, drops from the basis and x4 enters the 
basis. The new set of basic and nonbasic variables and their values are given as: 


so -|™ |-[%]-[$} x-[2]-[3] 


Blocking constraint 
> x1 20 


x leaves 


IL [0] , 


Figure 3.8. Blocking variable (constraint). 
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Moving from the old to the new basic feasible solution is illustrated in Figure 
3.8 in the original (x, x.) space. Note that as x4 increased by one unit, x 


decreases by 1/3 unit and x, decreases by 1/3 unit; that is, we move in the 
direction (-1/3, —1/3) in the (x, x2) space. This continues until we are blocked 
by the nonnegativity restriction x, > 0. At this point x, drops to zero and leaves 
the basis. 


3.6 TERMINATION: OPTIMALITY AND UNBOUNDEDNESS 


We have discussed a procedure that moves from one basis to an adjacent basis, 
by introducing one variable into the basis, and removing another variable from 
the basis. The criteria for entering and leaving are summarized below: 

|. Entering: x, may enter if z, — c, >0 


2. Leaving: xg. may leave if 


b, bj 

— = minimum,—: yz > 0. 
Vrk lsism | Vik 

Two logical questions immediately arise. First, what happens if each 


nonbasic variable x j hasz;-c; S$ 0? In this case, as seen by Equation (3.6), 


we have obtained an optimal basic feasible solution. Second, suppose that 
Zp — Cy > 0, and hence x; is eligible to enter the basis, but we cannot find any 


positive component y,,, that is, y,; <9. In this case, the optimal objective value 
is unbounded. These cases will be discussed in more detail in this section. 
Termination with an Optimal Solution 


Consider the following problem, where A is an m x n matrix with rank m. 


Minimize cx 
subject to Ax=b 
x>0. 


a. . : < A : ‘ * -l 
Suppose that x is a basic feasible solution with basis B; that is, x = P 0 a 


Let z” denote the objective of x”, that is, z” =c,B'b. Suppose further that 


z;-¢,S 0 for all nonbasic variables, and hence there are no nonbasic variables 


that are eligible to enter the basis. Let x be any feasible solution with objective 
value z. Then from Equation (3.3) we have 


Za7= 2 (ape pay (3.13) 
jeJ 
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Because z yreeS 0 and x ie 0 for all variables, then z” < z, and so, as in 


Equation (3.6), x” is an optimal basic feasible solution. 


Unique and Alternative Optimal Solutions 


We can get more information from Equation (3.13). If z pags 0 for all 
nonbasic components, then the current optimal solution is unique. To show this, 
let x be any feasible solution that is distinct from x”. Then there is at least one 


nonbasic component x, that is positive, because if all nonbasic components are 


zero, x would not be distinct from x”. From Equation (3.13) it follows that z > 


z", and hence, x” is the unique optimal solution. 


Now, consider the case where z oS 0 for all nonbasic components, 


but z, — cy, = 0 for at least one nonbasic variable x,. As x, is increased, we 


get (in the absence of degeneracy) points that are distinct from x” but have the 
same objective value (why?). If x, is increased until it is blocked by a basic 
variable (if it is blocked at all), we get an alternative optimal basic feasible 
solution. The process of increasing x, from level zero until it is blocked (if at 
all) generates an infinite number of alternative optimal solutions. 


Example 3.6 


Minimize -3x, + Xx 


subjectto x + 2x, + x3 = 4 
—Xy} + X + x4 = ] 
x}, X25 X3; X4 = 0 


Consider the basic feasible solution with basis B =[a,,aq]= E | and B™! 


= i i: The corresponding point is given by 


ro-[a}-e rf L-[  »-[2)-[9) 


and the objective value is -12. To see if we can improve the solution, calculate 
Zy —Cz and 23 — c3 as follows, noting that c,B | = (-3, 0), where ez =(-3, 
0): 


Z2-€2 = cpB ‘a, —c) =(-3,0)| 4 ]-1 =-7, 
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23 -¢; =¢,B'a3 —c3 = -3.0)] 9] £9533, 


Since both z2 — ce, <0 and z3; — c3 <0, then the basic feasible solution (x, 
X2, %3, x4) = (4, 0, 0, 5) is the unique optimal point. This unique optimal 
solution is illustrated in Figure 3.9a. 

Now, consider a new problem where the objective function -2x, — 4x 
is to be minimized over the same region. Again, consider the same point (4, 0, 0, 
5). The objective value is -8, and the new cp = (-2, 0), cpB! = (-2, 0). 


Calculate z,—c and z3 — c3 as follows: 
22 —-C7 = cpB ‘a, Cy = -2,0)] 7]+ 4= 0, 
23 -C, =¢gB a3 —c3 = (-2.0)] § |- 0=-2. 
In this case, the given basic feasible solution is optimal, but it is no longer a 


unique optimal solution. We see that by increasing x, a family of optimal 
solutions is obtained. Actually, if we increase x, keep x, = 0, and modify x, 


B ~ B'b-[B''a, |x, 


and x4, we get 


x 4- 2X9 
. x 
For any x2 < 5/3, the solution = vi 


Family of alternative 
optimal solutions 


(b) 
Figure 3.9. Termination criterion: (a) Unique optimal solution; 
(b) Alternative optima. 
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is an optimal solution with objective value —8. In particular, if x. = 5/3, we get 
an alternative basic feasible optimal solution, where x, drops from the basis. 


This is illustrated in Figure 3.9b. Note that the new objective function contours 
are parallel to the hyperplane x; + 2x2 = 4 corresponding to the first constraint. 


That is why we obtain alternative optimal solutions in this example. In general, 
whenever the optimal objective function contour contains a face of dimension 
greater than 0, we will have alternative optimal solutions. 


Unboundedness 
Suppose that we have a basic feasible solution of the system Ax = b, x > 0, with 
objective value zy. Now, let us consider the case when we find a corresponding 


nonbasic variable x, with z,—c, >0 and y, <0. This variable is eligible to 


enter the basis, since increasing it will improve the objective function value. 
From Equation (3.3) we have 


Z= Zo — (Zp ~-Cy) Xp. 
Since we are minimizing the objective z and since z, — c, > 0, then it is to our 
benefit to increase x, indefinitely, which will make z go to —oo. The reason that 
we were not able to do this before was that the increase in the value of x, was 


blocked by a basic variable. This puts a “ceiling” on x, beyond which a basic 
variable will become negative. But if blocking is not encountered, there is no 
reason why we should stop increasing x;,. This is precisely the case when y; < 
0. Recall that from Equation (3.8) we have 


~1 
XR= Bb -[yx |x 
and so, if y,; <0, then x, can be increased indefinitely without any of the basic 


variables becoming negative. Therefore, the solution x (where xp =B'b- 
[yu lee» the nonbasic variable x, is arbitrarily large, and other nonbasic 
variables are zero) is feasible and its objective value is z = Zo —(zg —Cy Xz, 
which approaches —oo as x, approaches ©, 

To summarize, if we have a basic feasible solution with z,—c, > 0 for 
some nonbasic variable x,, and meanwhile y, < 0, then the optimal objective 


value is unbounded, tending to -oo. This is obtained by increasing x; 


indefinitely and adjusting the values of the current basic variables, and is 
equivalent to moving along the ray: 
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Bb -Yx 
0 0 
: + Xz : Xz >0 
0 1 
0 0 


-1 
Note that the vertex of the ray is the current basic feasible solution P 0 ») and 


the direction of the ray is 
Yk 


0 
where the | appears in the Ath position. It may be noted that 
ed =(€p, Cy )d =—Cpy, tcp =—-Zy+Cg. 


But cy, —z, <0 (because x, was eligible to enter the basis). Hence, ed < 0 for 
the recession direction d of the (nonempty) feasible region, which is the 
necessary and sufficient condition for unboundedness. In Exercise 3.52 we ask 
the reader to verify that d is indeed an (extreme) direction of the feasible region. 


Example 3.7 
(Unboundedness) 
Minimize -—x, — 3x2 
subjectto x - 2x, < 4 
7X + x2 < 3 
XM xX, 2 0 


The problem, illustrated in Figure 3.10, clearly has an unbounded optimal 
objective value. After introducing the slack variables x3 and x4, we get the 


1-2 1 0 


constraint matrix A = E 101 


| Now, consider the basic feasible 


solution whose basis B = [a3,a4] = E at 


o-[2]-[3 LS}-L) =»-[2]-[2} 
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Increase xy 
(unblocked) 


Optimal objective value 
is unbounded along 
this ray 


Increase x> 
(blocked by x4) 


x] 


iy 


Figure 3.10. Unbounded optimal objective value. 


c = (-1,-3) 


Calculate z,-—c¢, and zz — cp as follows, noting c ,B! = (0, 0): 
24 - cy = cpB a, —cy =-c) = 1, 
22 -C2 = cpB la, —C2 =—-C2 = 3. 


So, we increase x7, which has the most positive z pep Note that xp = B b— 


Ea lL} Ta» 


The maximum value of x, is 3, at which instant x, drops to zero. Therefore, 


[B~!a,]x, and hence, 


the new basic feasible solution is (x; x2, x3, x4) = (0, 3, 10, 0). The new 
basis B is [a3,a2] = i 4 with inverse E it: Since cp =(0,—-3), we get 


cpB ! = (0, -3), and we calculate Z,—¢, and z4—cy, as follows: 


2) -c) = cpB a, “Qs (0, | 4] = 4, 


24-4 = c,B !a, —C4 = (0,-3)|| —-0=-3. 


Note that z; —c, > 0 and y, = Bla, = = < HI Therefore, the optimal objec- 


tive value is unbounded. In this case, if x, is increased and x4 is kept zero, we 
get the following solution: 


XzB= B'b-[B la, |x 
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x3] _]10)_|—l}, _|10+% 

x4 “| 3 -] 17 3+ x 
with x, 20, and x4 =0. Note that this solution is feasible for all x, = 0. In 
particular, 


Xy — 2X7 + x3 =X, —- 218 4+x,)+00+x,) = 4, 


and 


—Xy tX_ +X4 =—-Xy +(B+x,)+0=3. 


Furthermore, z = -9 ~ 4x,, which approaches -co as x, approaches oo. 
Therefore, the optimal objective value is unbounded by moving along the ray 


{(0, 3, 10,0) + x, (1,1, 1,0): x, > 0}. 


Again, note that the necessary and sufficient condition for unboundedness holds; 
in particular, 


ed = (1, ~3, 0, 0) =-4<0. 


1 
1 
1 
0 


3.7 THE SIMPLEX METHOD 


All the machinery that is needed to describe the simplex algorithm and to prove 
its convergence in a finite number of iterations (in the absence of degeneracy) 
has been generated. Given a basic feasible solution and a corresponding basis, 
we can either improve it if z, —c, > 0 for some nonbasic variable x,, or stop 


with an optimal point if z;—c, < 0 for all nonbasic variables. If z, -—c, > 0 


and the vector y, contains at least one positive component, then the increase in 
x, will be blocked by one of the current basic variables, which drops to zero 
and leaves the basis. On the other hand, if z, -c, > Qand y,;, <0, then x, can 


be increased indefinitely, and the optimal objective value is unbounded (—oo). 
This discussion is precisely the essence of the simplex method. 

We now give a summary of the simplex method for solving the following 
linear programming problem: 


Minimize ex 
subjectto Ax=b 
x>0, 


where A is an m X m matrix with rank m (the requirement that rank (A) = m will 
be relaxed in Chapter 4). 
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The Simplex Algorithm (Minimization Problem) 


INITIALIZATION STEP 


Choose a starting basic feasible solution with basis B. (Several procedures for 
finding an initial basis will be described in Chapter 4.) 


MAIN STEP 


1. 


Solve the system Bx, = b (with unique solution xz = B'b= b). 
Let Xp b, Xy = 0, and z= CpXxp. 


Solve the system wB = cz (with unique solution w = c 3B"); (The 


vector w is referred to as the vector of simplex multipliers because its 
components are the multipliers of the rows of A that are added to the 
objective function in order to bring it into canonical form.) Calculate 


zj;—¢; = wa, — c, for all nonbasic variables. (This is known as the 


pricing operation.) Let 


Zy — Cy = Maximum{z ; —c ;} 
jeJ ae 


where J is the current set of indices associated with the nonbasic 
variables. If z,—c, <0, then stop with the current basic feasible 


solution as an optimal solution. Otherwise, go to Step 3 with x, as 


the entering variable. (This strategy for selecting an entering variable 
is known as Dantzig’s Rule.) 


Solve the system By, = a, (with unique solution y, = Bla, ). If 
y, <0, then stop with the conclusion that the optimal solution is 
unbounded along the ray 


[eles] 4 


where e, is an (” — m)-vector of zeros except for a 1 at the Ath 
position. If y, < 0, go to Step 4. 
Let x, enter the basis. The index r of the blocking variable, x, , 


which leaves the basis is determined by the following minimum ratio 
test: 


Be ek b; 
‘= minimum, ——: yz >0>. 
Vrk I<ism Vik 

Update the basis B where a, replaces ap , update the index set J, 


and repeat Step 1. 
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Modification for a Maximization Problem 


A maximization problem can be transformed into a minimization problem by 
multiplying the objective coefficients by —1. A maximization problem can also 


be handled directly as follows. Let z, —c, instead be the minimum z; —c; for / 


nonbasic; the stopping criterion is that z, —c, > 0. Otherwise, the steps are as 
previously presented. 


Finite Convergence of the Simplex Method in the Absence of Degeneracy 


Note that at each iteration, one of the following three actions is executed: (a) We 
may stop with an optimal extreme point if z, —c, <0; (b) we may stop with an 


unbounded solution if z,—c, >0 and y, < 0; or else, (c) we generate a new 
basic feasible solution if z; —c, >0Oand y,; X 0. In the absence of degeneracy, 


b, > 0, and hence x, = b,/y,, > 0. Therefore, the difference between the 
objective value at the previous iteration and that at the current iteration is 
x4(Zy —Cz) > 0. In other words, the objective function strictly decreases at each 


iteration, and hence the basic feasible solutions generated by the simplex 
method are distinct. Since the number of basic feasible solutions is finite, the 
method must stop in a finite number of steps with either an optimal solution or 
with an indication of an unbounded optimal objective value. From this 
discussion the following theorem is evident. 


Theorem 3.5 Finite Convergence 


In the absence of degeneracy (and assuming feasibility), the simplex method 
stops in a finite number of iterations, either with an optimal basic feasible 
solution or with the conclusion that the optimal objective value is unbounded. 


In the presence of degeneracy, however, as we have mentioned, there is 
the possibility of cycling in an infinite loop. This issue is discussed at length in 
Chapter 4. 


Example 3.8 
Minimize —x,; — 3x, 
subjectto 2x, + 3x, < 6 
xX aa PB) < |] 
Xx, x4 = 0. 


The problem is illustrated in Figure 3.11. After introducing the nonnegative 
slack variables x3 and x4, we get the following constraints: 


2x, + 3xp + %3 = 6 
—X + XxX + xy = 1 
xX, Xx), x3, %4 > 0. 
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Figure 3.11. Example of the simplex method. 

Iteration 1 

Let B = [a3, a4] = E ‘1 and N = [a), a2] = E i Solving the system Bx, 
=b leads to xg = x3 = 6 and xp, = x4 = 1. The nonbasic variables are x and x, 
and the objective z = ¢pxz = (0, 0/4] = 0. In order to determine which 


variable enters the basis, calculate a =¢,B"'a, —C; =Wwa;—C;. First we 


find w by solving the system WB = cp: 


Gn.m)| 4 7 |=@.0) =» =W =0. 


Hence, 
2) —C; = Wa) -c) = 1, 


2 —Cp = Wan —-C2 =3. 


Therefore, x, is increased. In order to determine the extent by which we can 


increase x7, we need to calculate y, by solving the system By» = ap: 


1 0 3 
E 22 |-[f] 22-3 and yon = 1. 


The variable XB, leaving the basis is determined by the following minimum 


ratio test: 


b 
Minimum ete = minimum is ; =]. 
Yi2 22 
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Therefore, the index r = 2; that is, xg, = x4 leaves the basis. This is also 


2 | : BI 7 Hi - He (3.14) 


and x4 first drops to zero when x2 = 1. 


obvious by noting that 


Iteration 2 


The variable x, enters the basis and x4 leaves the basis: 
1 3 2 0 
B = [a3,a)] = F i and N = [a),a4] = 5 i}: 


Now, xg can be determined by solving Bx, = b or simply by noting that x» = 
1 in Equation (3.14): 


XB, _|%3] _ 3 Nf 0 
XB) 2 x4 ~ TL? X4 ~ 107 
The objective value is z = —3. Calculate w by wB = cp: 


(mn) g 1 |=@.-3)= mi =0 and Ww = -3. 


Hence, 
Zz} —c = wa; -C 


= ,-3)| 5 | ‘1 S4 


The variable x4 left the basis in the previous iteration and cannot enter the basis 
in this iteration (because z4—c4 < 0; see Exercise 3.35). Therefore, x, is 
increased. Solve the system By, = a): 


12 2 
E ez ]-[4]son-s and y, =-l. 


Since y2, <0, then xg = x3 leaves the basis as x, is increased. This is also 


clear by noting that 


and x3 drops to zero when x, = 3/5. 
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Iteration 3 


Here, x, enters the basis and x3 leaves the basis: 


B = [a),a)] = E 1 and =-N = [a3, a4] = lo "| 


XB, c2') ap _ | x3} _|0 
me |-L2]-[s} »-[2]-[oh 
The objective value is given by z = —27/5. Calculate w by wB = cp: 


(myo) 3 ‘ =(-1,-3)> w =-4/5 and w, =-3/5. 


The variable x3 left the basis in the last iteration and cannot enter the basis in 


this iteration (because z, —c; <0). Hence, we compute 


Z4—Cq = Wag —Cq 
0 
=(-4/5, -2/8) (]}-0=-3/5 


Therefore, z;—c,; < 0 for all nonbasic variables, and the current point is 


iOS 
optimal. The optimal solution is therefore given by 


(%1, X2, 23,4) = (3/5, 8/5, 0, 0) 


with objective value —27/5. Figure 3.11 displays the simplex path, that is, the 
sequence of steps that the simplex method took to reach the optimal point. 


3.8 THE SIMPLEX METHOD IN TABLEAU FORMAT 


At each iteration, the following linear systems of equations need to be solved: 
Bx, =b, wB= cp, and By; = a,. Various procedures for solving and updating 


these systems will lead to different algorithms that all lie under the general 
framework of the simplex method described previously. In this section, we 
describe the simplex method in tableau format. In subsequent chapters we shall 
discuss several procedures for solving the preceding systems for general linear 
programs as well as for problems having special structures such as network flow 
problems. 


Suppose that we have a starting basic feasible solution x with basis B. 
The linear programming problem can be represented as follows. 
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Minimize z 


subjectto z — Cpxp -— CyXy = O (3.15) 
Bx, Ae Nxy = b (3.16) 
XB> Xn = 0. 
From Equation (3.16) we have 
xp + BINxy = Bb. (3.17) 
Multiplying (3.17) by ¢p and adding to Equation (3.15), we get 
z+O0xpz + (¢gB 'N-cy)xy =¢pB 'b. (3.18) 


Currently, x, = 0, and from Equations (3.17) and (3.18) we get xp = Bb 


andz=c 3B !b. Also, from (3.17) and (3.18) we can conveniently represent the 
current basic feasible solution with basis B in the following tableau. Here, we 
think of z as a (basic) variable to be minimized. The objective row will be 
referred to as row 0 and the remaining rows are rows | through m. The right- 
hand-side column (RHS) will denote the values of the basic variables (including 
the objective function). The basic variables are identified in the far left column. 


Row 0 


Rows | through m 


The tableau in which z and xg have been solved in terms of xy is said to be in 
canonical form. Not only does this tableau give us the value of the objective 
function ¢ pB 'b and the basic variables B-'b in the right-hand-side column, 
but it also gives us all the information we need to proceed with the simplex 
method. Actually the cost row gives us cpB'N-c nw» which consists of the 
(z; —¢;)-values for the nonbasic variables. So, row zero will tell us if we are at 


an optimal solution (if each z ree Ss 0), and which nonbasic variable to 


increase otherwise. If x, is increased, then the vector y, = B a,, which is 
stored in the tableau in rows | through m under the variable x,, will help us 
determine by how much x, can be increased. If y, < 0, then x; can be 
increased indefinitely without being blocked, and the optimal objective value is 
unbounded. On the other hand, if y, < 9, that is, if y, has at least one positive 
component, then the increase in x, will be blocked by some currently basic 
variable, which drops to zero. The minimum ratio test (which can be performed 
since B'b = b and y, are both available in the tableau) determines the 
blocking variable. We would like to have a scheme that will do the following: 
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1. Update the basic variables and their values. 
2. Update the (z; —c;)-values of the new nonbasic variables. 


3. Update the y ; columns. 
Pivoting 


All of the foregoing tasks can be simultaneously accomplished by a simple 
pivoting operation. If x, enters the basis and xg leaves the basis, then 
pivoting on y,, can be stated as follows: 
1. Divide rowrby y,,. 
2. For i= 1,..., m and i # r, update the ith row by adding to it —y, 
times the new rth row. 
3. Update row zero by adding to it c, —z, times the new rth row. The 


two tableaux of Tables 3.1 and 3.2 represent the situation 
immediately before and after pivoting. 


Table 3.1. Before Pivoting 
Z Xp Xp, xB, x; Xp RHS 


Let us examine the implications of the pivoting operation. 
1. The variable x, entered the basis and xg left the basis. This is 
recorded in the left-hand-side of the tableau by replacing xg, 
with x,. For the purpose of the following iteration, the new xg is 


now X;. 


2. The right-hand-side of the tableau gives the current values of the 
basic variables (review Equation (3.10)). The nonbasic variable 
values are zero. 


3. Suppose that the original columns of the new basic and nonbasic 
variables are B and N, respectively. Through a sequence of 


elementary row operations (characterized by pivoting at the 
intermediate iterations), the original tableau reduces to the current 


tableau with B replaced by I. From Chapter 2 we know that this is 


equivalent to premultiplication by B!. Thus pivoting results in a 
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new tableau that gives the new B"! N under the nonbasic variables, 
an updated set of (z 7 = c; )-values for the new nonbasic variables, 


and the values of the new basic variables and objective function. 
The Simplex Method in Tableau Format (Minimization Problem) 


INITIALIZATION STEP 


Find an initial basic feasible solution with basis B. Form the following initial 
tableau: 


XB 


MAIN STEP 
Let z, —c, = maximum {z;-c,;: j €J}. If z, —c, <0, then stop; the current 


solution is optimal. Otherwise, examine y,. If y, <0, then stop; the optimal 
objective value is unbounded along the ray 


where e,; is a vector of zeros except for a | at the Ath position. If y, % 0, 
determine the index r as follows: 


ar bj 
r= sian Viz > o. 
Vrk Sis — | Yig 


Update the tableau by pivoting at y,,. Update the basic and nonbasic variables 
where x, enters the basis and xg _ leaves the basis, and repeat the main step. 


Example 3.9 
Minimize x + X%) — 4x3 
subjectto x + XX + 2x3 S 9 
Hy + X% - x S$ 2 
—Xy + XxX + X3 < 4 
4, x2, x3 > 0. 


Introduce the nonnegative slack variables x4, x5, and xg. The problem becomes 
the following: 
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Minimize x + X%, — 4x, + Ox, + Oxs + OX¢ 

subjectto x + X, + 2x, + X4 =9 
X +t X -— XB + Xs = 2 
“xy + % + % + X% = 4 
xX, Xp, x3, X4, Xs, Xe 2 0. 


Since b > 0, then we can choose our initial basis as B = [a4,a5,a¢]=I,, and we 


indeed have B"'b = b > 0. This gives the following initial tableau: 


Iteration 1 


Iteration 2 


Iteration 3 


This is an optimal tableau since z;-—c '; <9 for all nonbasic variables. The 


j 
optimal solution is given by 


x = 1/3, X2 =0, x3 = 13/3, 
with z= —17. 


Note that the current optimal basis consists of the columns a,, a5, and a3, 
namely, 
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B = [a),a5,a3] =| 1 I —l)}. 

0 1 


Interpretation of Entries in the Simplex Tableau 


Consider the following typical simplex tableau and assume nondegeneracy: 


The tableau may be thought of as a representation of both the basic variables xp 
and the cost variable z in terms of the nonbasic variables x,y. The nonbasic 
variables can therefore be thought of as independent variables, whereas xp and 
z are dependent variables. From row zero, we have 


Zz =¢pB 'b—(cgB 'N-cy)xy 
=cgB'b+ ¥ (¢;-2;)x;, 
jeJd 


and hence, the rate of change of z as a function of any nonbasic variable x ,, 
namely 6z/6x;, is simply c;~z,. In order to minimize z, we should increase 
x; if 0z/ Ox i < 0, that is, if z Pie ee 0. (Here and later, each partial derivatives 


is defined in the usual rate of change sense, holding all the other independent 
variables constant at their current values.) 

Also, the basic variables can be represented in terms of the nonbasic 
variables as follows: 


xp = B'b-B'Nxy 


Bb- > [Bola, |x; 
jeJ 

= B!b- x [y; ]x;- 

jeJ 


Therefore, 0x p/Ox j = ~y,; that is, -y j is the rate of change of the basic 
variables as a function of the nonbasic variable x j- In other words, if x '; increases 
by one unit, then the ith basic variable Xp, decreases by an amount yj, or 


simply, xp. /Ox; = —yy. The column y,; can be alternatively interpreted as 


132 Chapter 3 


follows. Recall that By ; =a;, and hence y, represents the linear combination 


of the basic columns that are needed to represent a ;. More specifically, 


The simplex tableau also gives us a convenient way of predicting the rate of 
change of the objective function and the value of the basic variables as a 
function of the right-hand-side vector b. Since the right-hand-side vector 
usually represents scarce resources, we can predict the rate of change of the 
objective function as the availability of the resources is varied. In particular, 


z=c,B 'b- > (z;-C;)x; 
jeJ 


and hence, 0z/0b=c ga If the original identity consists of slack variables 
having zero costs, then the elements of row zero at the final tableau under the 


slacks give cB '1-0 = cpB |, which is 6z/0b. More specifically, if we let 


w= cpB |, then 0z/0b; =w;. 
Similarly, the rate of change of the basic variables as a function of the 
right-hand-side vector b is given by 
Op _ Bt 
ob 


In particular, OX. /Ob is the ith row of Bo, Ox p/ 0b H is the jth column of Bo, 


and Ox, / Ob; is the (i, /) entry of BT. 


Note that if the tableau corresponds to a degenerate basic feasible 


solution, then as a nonbasic variable x j actually increases, at least one of the 


basic variables may become immediately negative (see Equation 3.8), 
destroying feasibility. In this case, the indicated rate of change in the objective 


value with respect to x; is not realizable, because when x; increases, holding 


the other nonbasic variables at zero, the resulting points are infeasible. Similar 
remarks hold for the other partial derivatives. In particular, if 0z/0b; = w;, at 


optimality, then although this rate of change is valid in the usual sense of the 
partial derivative, it may not be the actual realizable change in the optimal 
objective function value when 8; is increased in the case of degeneracy. As we 


shall be seeing later in Chapter 6, the objective function z as a function of }; 
may not be differentiable in such a case, and so w; may only be the slope of one 
possible tangential support to this function at the current value of 5;. 


Identifying B"! from the Simplex Tableau 


The basis inverse matrix can be identified from the simplex tableau as follows. 
Assume that the original tableau has an identity matrix. The process of reducing 
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the basis matrix B of the original tableau to an identity matrix in the current 
tableau is equivalent to premultiplying rows | through m of the original tableau 


by B! to produce the current tableau (why?). This also converts the identity 


matrix of the original tableau to B"!. Therefore B™! can be extracted from the 


current tableau as the submatrix in rows 1 through m under the original identity 
columns. 


Example 3.10 


To illustrate the interpretations of the simplex tableau, consider Example 3.9 at 
iteration 2. Then 


ee 

Oxy x2 OX6 

Ox4 Ox Ox3 -| 

Ox Oxy ; OX6 , 
1 

OXp -|_2 ; 

Oxy | -1 

OE OO Gz 4 

Ob, Ob, ° Obs 

0x5 i Ox4, =-2, 

Ob, Ob; 


and 


The vector ay can be represented as a linear combination of the basic columns 


as follows: a, =— lay + 2a5 + a3. At iteration 3, we have 
LG. 
Ox OX6 : 
Ores Gao. Se so 
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See Sh. ee, 
aby ab, ab, 
23/3 
Oey) * 34 
F) Lj 
a 1/3 
Oe) Oa: aig, 
é 3’ Ob, 
and 
3 0 -2/3 
B!'=| 0 1 1 
/3 0 13 


The vector az can be represented as a linear combination of the basic columns 
as follows: ay =—(1/3)a, + 2a5 + (2/3)a3. 


3.9 BLOCK PIVOTING 


Throughout this chapter we have considered the possibility of entering a single 
nonbasic variable into the basis at each iteration. Recall that whenever a 
nonbasic variable enters the basis we must ensure that the new set of variables, 
the current basic set minus the exiting variable plus the entering variable, 
satisfies the following: (1) it also forms a basis; (2) it remains feasible, that is, 
Xp 20 for all i; and also, we must have that (3) the value of the objective 


function either remains constant or decreases (for a minimization problem). It is 
possible to enter sets of nonbasic variables as long as the same three conditions 
are satisfied. The process of introducing several nonbasic variables simulta- 
neously is called block pivoting. However, in the case of multiple entries, the 
foregoing conditions become harder to ensure. 

Suppose that we enter several nonbasic variables into the basis in such a 
way that Condition 2 is maintained. Note that 


z=cpB 'b- Ej ~¢,)x;- 
JE 


j7¢j > 0 for all entering 


variables, we shall ensure that the value of z will either remain constant or else 
decrease. 


If we, for example, use the entry criterion that z 
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With respect to the question whether the new set still forms a basis, we 
must extend the rule that the pivot element be nonzero. Consider the basic 
equations before pivoting: 


Xp= B 'b-B'Nxy. 


Ro! _Pp-! _| XB, _| XM 
Let b=Bb, Yy =B'N, xp = »Xy= where the vector x,y 
Xp XN, 1 


enters and the vector xg, leaves the basis. Here xp, and xy, each contain the 


same number of variables (why?). On partitioning the basic system, we get 


E | XB, -|2 iF Yu, ¥mp |f ¥™ 
0 bls, by} L¥ va, Ying, | ¥a2 J 
On rearranging, we get 


He ¥wi, |f *3, -|> |} 0 Ym, | *2 
0 Yn, XN, b, I, Yn25 XN> 
Now, the new set of variables, x B and x Np will form a basis if and only if the 


I, Yy Bi a . 
matrix 'l | can be converted into the identity matrix via row operations; 


0 Yy,, 
that is, if this matrix is invertible. From Chapter 2 we know that this matrix is 
invertible if and only if the determinant of the square matrix Yn, . is nonzero. 


This is a natural extension of the rule when entering only one variable. The new 
rule for maintaining a basis is as follows. Consider the square submatrix formed 
by the elements in the leaving rows and entering columns of the current tableau. 
If the determinant of this submatrix is nonzero, the new set will form a basis. 

Rules for checking feasibility of the new basic set are more involved. 
Except in special circumstances, such as network flows, the rules for feasibility 
are difficult to check. This is primarily the reason why block pivoting is 
generally avoided in practice. 


EXERCISES 


[3.1] Consider the following linear programming problem: 


Maximize x + 2x, 
subjectto x1 - 4x, < 4 
—2x + x2 < 2 
—3x at 4x, < 12 
2x} + x2 < 8 
X> x2 2 0 
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a. Sketch the feasible region in the (x,, x») space and identify the 


optimal solution. 

b. Identify all the extreme points and reformulate the problem in terms 
of convex combination of the extreme points. Solve the resulting 
problem. Is any extreme point degenerate? Explain. 

c. Suppose that the fourth constraint is dropped. Identify the extreme 
points and directions and reformulate the problem in terms of convex 
combinations of the extreme points and nonnegative linear combina- 
tions of the extreme directions. Solve the resulting problem and 
interpret the solution. 

d. Is the procedure described in Parts (b) and (c) practical for solving 
larger problems? Discuss. 


[3.2] Consider the following constraints: 


2x, + 3x5 < 6 
2x, E x2 < 2 
xX) -— 2x < 0 
x1, x2 > 0. 


a. Draw the feasible region. 
b. Identify the extreme points, and at each extreme point identify all 
possible basic and nonbasic variables. 


c. Suppose that a move is made from the extreme point 3] to the 


extreme point H in the (x,, x.) space. Specify the possible enter- 


ing and leaving variables. 
[3.3] Consider the following problem: 


Maximize xX + 3X, 
subject to Xx ~ 2x% Ss O 
2x; + x2 < 4 
5x, + 3X2 < 15 
X15 xX, 2 0. 


a. Solve this problem graphically. 
b. Solve the problem by the simplex method. 


[3.4] Solve the following problem by the simplex method starting with the basic 
feasible solution corresponding to the vertex (xj, x.) = (0, 2). Sketch the 
feasible region in the nonbasic variable space. 


Maximize 2x, - x9 

subjectto 2x, + 3x, = 6 
xX) — 2x << O 
xX, xX, 2 0 


(Hint: Identify the initial basis and find its inverse.) 
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[3.5] Solve the following linear programming problem by the simplex method. 
At each iteration, identify B and BT. 


Maximize 3x, + 2x, + 3 


subjectto 3x, - 3x, + 2x3 < 3 
—x} + 2x, + % S< 6 
X15 X9, % «2 0 
[3.6] Consider the following problem: 
Maximize —3x, — 2x 
subjectto -x) + Xx» < i 
2x) + 3x, < 6 
x = 0 
x2 2 4/ 3. 


Solve the problem graphically. 
Set up the problem in tableau form for the simplex method, and 
obtain an initial basic feasible solution. 
c. Perform one pivot. After one pivot: 
i. Indicate the basis matrix. 
ii. Indicate the values of all variables. 
iii. Is the solution optimal? 
d. Draw the requirements space representation. 
i. Give the possible bases. 
ii. Give the possible feasible bases. 
e. Relate each basis in Part (d)(i) to a point in Part (a). 


[3.7] Consider the following linear programming problem: 


oS 


Maximize 2x, + % - 4% 

subjectto 2x, + x, + 4%, < 6 
x + 4x, = x3 < 4 
xX); X2; x3 = 0. 


Find an optimal solution by evaluating the objective function at all extreme 
points of the constraint set. Show that this approach is valid in this problem. 
Now, suppose that the first constraint is replaced by x, +x, —4x3 <6. Can the 


same approach for finding the optimal point be used? Explain why. 


[3.8] Consider a maximization linear programming problem with extreme points 
X}, Xj, X3, and x4, and extreme directions d), dj, and d3, and with an 


objective function gradient ¢ such that ex) =4, ex, =6, ex3 =6, cx4 =3, ed, 
= 0, ed, = 0, and cd3 = —2. Characterize the set of alternative optimal solutions 
to this problem. 

[3.9] Consider the following linear programming problem: 


Maximize 2x, + 2x, + 4x3 + Sxg + 3X6 
subjectto 3x, + 6x, + 3x3 + 3xy + 3x5 + 4x_ < 60 
xX; X25 x3; X45 X55 X6 = 0. 
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This problem has one constraint in addition to the nonnegativity constraints, and 
is called a knapsack problem. Find all basic feasible solutions of the problem, 
and find an optimum by comparing these basic feasible solutions. Hence, 
prescribe a general rule to solve a knapsack problem of the type to Maximize 


n n 
YX CjXj subject to + a jx; <b, x; 20 for j = 1,...,2, where b> 0, c; 20, VJ, 
j=l j=l 


and a; > 0, Vj, based on the ratios cj /aj, j = 1,...,2. Comment on how you 


would treat the case ers 0 for any /, or cj; > O and a; <0 for any /. 


[3.10] Consider the polyhedral set consisting of points ( x,,x ) such that 


x} + 2Xx> < 2 


xX, X unrestricted. 


Verify geometrically and algebraically that this set has no extreme points. 
Formulate an equivalent set in a higher dimension where all variables are 
restricted to be nonnegative. Show that extreme points of the new set indeed 
exist. 

[3.11] Consider the linear program: Minimize ex subject to Ax < b, x => 0, where 
¢ is a nonzero vector. Suppose that the point xq is such that Axg <b and xo > 


0. Show that xg cannot be an optimal solution. 


[3.12] Consider the following system: 


x + 2X + x3 < 3 
—2x, + 2x, + 2% < 3 
X15 X95 x3 > 0 


The point (1/2, 1/2, 1/2) is feasible. Verify whether it is basic. If not, use the 
method described in the text for reducing it to a basic feasible solution. 


[3.13] Answer the following questions along with a concise explanation with 
respect to the linear program to maximize ex subject to xe X = {x: Ax=b,x> 
0}, where A is m x n of rank m <n. 


a. Inasimplex tableau, if z prepa ~7 for a nonbasic variable x 7 what 


is the change in objective value when x; enters the basis given that 


the minimum ratio is 3 in the pivot? 


b. Ifan extreme point is optimal, then is it possible that not all z p7eye 


0 for an associated basis? 

c. If there exists a d such that Ad = 0, d > 0, and cd > 0, then is the 
optimal objective value unbounded? 

d. Let x be a feasible solution with exactly m positive components. Is 
X necessarily an extreme point of X? 

e. Ifanonbasic variable x, has z, —c, = 0 at optimality, then can one 


claim that alternative optimal solutions exist? 
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f. If x, and x» are adjacent points and if B, and By are respective 


associated bases, then these bases are also adjacent. True or false? 
Explain. 

g. Is it possible for an optimal solution to have more than m positive 
variables? 

h. Suppose that n = m+ 1. What is the least upper bound on the number 
of extreme points and feasible bases? 

i. A p-dimensional polyhedron can have at most p extreme directions. 
True or false? Explain. 

j. Let x be an extreme point having (m — 1) positive components. 
Then there are (p + 1) bases associated with this extreme point, 
where p = n — m. True or false? (Assume that Ax = b does not imply 
any variable to be a constant.) Explain. 


[3.14] Consider the region defined by the constraints Ax > b, where A is an m x 
n matrix with m > n. Further suppose that rank (A) = ”. Show that xq is an 


extreme point of the region if and only if the following decomposition of A and 
b is possible: 


A A, | “rows = b, | rows 
~ | Ay |m—n rows ~ | by |m—n rows 


A)Xq =b), ArX9 2b, 


rank (A)) =n. 
[3.15] Let X = {x : Ax =b, x > 0) where A is m x n of rank m. Let x be a 
feasible solution such that x,...,x, are positive and x,4),..., X, are zero. 


Assuming that the columns aj,...,a, of A corresponding to the positive vari- 


ables are linearly dependent, construct feasible points x’ and x” such that x is a 
convex combination of these points. Hence, argue that if x is an extreme point of 
X, it is also a basic feasible solution. Conversely, suppose that x is a basic 
feasible solution of X with basis B and that x = Ax’ + (1 — 4)x" for some 0 <A 


<1land x’, x"e X. Denoting xg and x, as the corresponding basic and non- 
basic variables, show that x'y = x¥ =0 and x’ = x',=B'b. Hence, argue 
that x is an extreme point of X. 


[3.16] Corresponding to the sequence of simplex iterations discussed in Section 
3.4 with respect to the problem in Figure 3.5, draw on this figure the sequence 
of simplices considered by the simplex algorithm. 


[3.17] Consider the constraints Ax = b, x > 0 and assume that they form a 
bounded region. Consider the following two problems, where x,, is the nth com- 
ponent of x: 


Minimize Xi 


subject to Ax=b 
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Maximize Xp 
subject to Ax=b 
x>0. 


Let the optimal objective values of these two problems be x, and x), respec- 
tively. Let x, be any number in the interval [ x), x, ]. Show that there exists a 


feasible point whose nth component is equal to x,,. 


[3.18] Suppose that we have a basic feasible solution of the system Ax = b, x > 
0 with basis B. Suppose that z, —c, > 0 and x, is introduced into the basis and 


xB is removed from the basis. Denote the new basis by B. Show algebraically 
that after pivoting: 


a. The column under x; is (B)!a je 
b. The right-hand-side is (B)~'b. 


c. The new cost row is composed of (cg By! a;—C;. 


(Hint: Suppose that 


B = (81,.895...58;5--5Ajy) 
B = (a),.49,...,8¢5..-58y)- 


First show that B = BE, and (B)"! = E'B"!, where 


rth column 
0 Vk 0 
0 V2k 0 
EZ 
0 0 Vrk 0 |< rthrow 
0 0 See Ymk ae 1 


This form is usually called the product form of the inverse and is discussed in 
more detail in Section 5.1.) 


[3.19] Suppose that we have a basic feasible solution that is nondegenerate. 
Furthermore, suppose that an improving nonbasic variable enters the basis. 
Show that if the minimum ratio criterion for exiting from the basis occurs at a 
unique index, then the resulting basic feasible solution is nondegenerate. Is this 
necessarily true if the first solution is degenerate? 


[3.20] An agricultural mill produces feed for cattle. The cattle feed consists of 
three main ingredients: corn, lime, and fish meal. These ingredients contain 
three nutrients: protein, calcium, and vitamins. The following table gives the 
nutrient contents per pound of each ingredient: 
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INGREDIENT 
NUTRIENT CORN LIME FISH MEAL 
Protein 25 15 25 
Calcium 15 30 20 
Vitamins 5 12 8 


The protein, calcium, and vitamins content per pound of the cattle feed must be 
in the following intervals respectively: [18, 22], [20, 0), and [6, 12]. If the 
selling prices per pound of corn, lime, and fish meal are, respectively, $0.20, 
$0.08, and $0.50, find the least expensive mix. 


(Hint: First find a basis B such that B™'b > 0.) 


[3.21] A nut packager has on hand 150 pounds of peanuts, 100 pounds of 
cashews, and 50 pounds of almonds. The packager can sell three kinds of 
mixtures of these nuts: a cheap mix consisting of 90 percent peanuts and 10 
percent cashews; a party mix with 50 percent peanuts, 30 percent cashews, and 
20 percent almonds; and a deluxe mix with 20 percent peanuts, 50 percent 
cashews, and 30 percent almonds. If the 12—-ounce can of the cheap mix, the 
party mix, and the deluxe mix can be sold for $0.80, $1.10, and $1.30, 
respectively, how many cans of each type would the packager produce in order 
to maximize the return? 


[3.22] A firm makes three products, 1, 2, and 3. Each product requires produc- 
tion time in three departments as shown. 


PRODUCT DEPARTMENT 1 DEPARTMENT 2 DEPARTMENT 3 


1 3 hr/unit 2 hr/unit 1 hr/unit 
2 4 hr/unit 1 hr/unit 3 hr/unit 
3 2 hr/unit 2 hr/unit 3 hr/unit 


There are 500, 400, and 300 hours of production time available in the three 
departments, respectively. If products 1, 2, and 3 contribute $3, $4, and $2.5 per 
unit to profit, respectively, find an optimal product mix. 


[3.23] Solve Exercise 1.11 as a linear model by the simplex method. Find the 
actual value of the objective function 36/x2x3 corresponding to the optimal point 
obtained from the simplex method. By trial and error see if you can find a 
feasible point whose value for the objective function 36/x2x3 is better than that 


obtained previously. 


[3.24] Solve Exercise 1.10 to find the number of barrels of each crude oil that 
satisfies the demand and minimizes the total cost: (Hint: First find a basis B 


having B'b >0.) 
[3.25] Solve the following problem by the simplex method: 
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Maximize x, 2X, + x3 


subjectto x, + 2x, + 3x3; < 12 
xX + %X% -— % < 6 

—xX] =f 3x9 < 9 

X15 Xo, x3 2 0. 


[3.26] Use the simplex method to solve the following problem. Note that the 
variables are unrestricted in sign. 


Minimize 2x, - xX» 

subjectto x, - 3x, 2 -3 

2x4, + XX 2 -2 

2x + 3x4 < 6 

3x) - 2x, < 6. 

[3.27] Consider the following problem: 

Maximize 2x, + 2% + 6x; - 4x4 
subjectto x, + 2x, + 4x3 - Xx, S 6 
2x, + 3X, - x3 + XxX, S 12 
x] + x3 ~O«OF+ X4 < 2 
xX, xX, x3, xy 2 O 


Find a basic feasible solution with the basic variables as x, x7, and x4. Is this 
solution optimal? If not, then starting with this solution find an optimal solution. 
{3.28] Consider the following problem: 


Maximize —3x, + 2x, - x3; + X% 
subjectto 2x, - 3x, - x3 + 4x S O 
—xy) + 2X5 + 2x3 —- 3x, 4 < 1 
x, be x4 = 4x3 + X4 < 8 
xX; Xo, X3, xy 2 0. 


Use the simplex method to verify that the optimal objective value is unbounded. 
Make use of the final simplex tableau to construct a feasible solution having an 
objective of at least 3500. Make use of the final tableau to construct a direction 
d such that ed > 0. 


[3.29] Find a nonbasic feasible optimal solution of the following problem: 


Maximize 11x, + 2X, - x3 + 3x4 + 4x5 + X¢ 
subjectto 5x, + XxX - x3 + 2x4 + X5 = ]2 
—14x,; - 3x, + 3x3 - 5x4 +x = 2 
2x, + (1/2)x, - (1/2)x3, + (1/2)x4 < 5/2 
3x, + (1/2)x. + (1/2)x3 + (3/2)x4 < 3 
Xs xX, X35 X4, Xs, % 2 O90. 


(Hint: Let the initial basis consist of x5, xg and the slack variables of the last 
two constraints. Then find alternative optimal solutions for the problem.) 
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[3.30] The following mathematical formulation describes a problem of 
allocating three resources to the annual production of three commodities by a 
manufacturing firm. The amounts of the three products to be produced are 
denoted by x,, x2, and x3. The objective function reflects the dollar contribution 


to profit of these products. 


Maximize 10x, + 15x2 + 5x3 

subject to 2x, + x < 6000 
3x, + 3x. + x3 < 9000 
xy + 2x_ + 2x3 < 4000 
X, x2, x3 2 0. 


a. Without using the simplex method, verify that the optimal basis 
consists of the slack variable of the first constraint, x, and xp. 


b. Make use of the information in Part (a) to write the optimal tableau. 
c. The Research and Development Department proposes a new product 


whose production coefficients are represented by [2,4,2]'. If the 


contribution to profit is $15 per unit of this new product, should this 
product be produced? If so, what is the new optimal solution? 

d. What is the minimal contribution to profit that should be expected 
before production of this new product would actually increase the. 
value of the objective function? 


[3.31] Solve Exercise 1.12 by the simplex method. Suppose that extra man— 
hours can be obtained by allowing overtime at the average of $14 per hour. 
Would it be profitable to increase the man-hours? If so, by how much? How 
would this increase the profit? 


[3.32] Solve Exercise 1.15 by the simplex method. 


[3.33] Can a vector that is inserted at one iteration in the simplex method be 
removed immediately at the next iteration? When can this occur and under what 
entering variable choice rule would it be impossible? 


[3.34] We showed in the text that z 77o = 0 for a basic variable. Interpret 


this result. 
[3.35] Show that in the simplex method if a variable x j leaves the basis, it 


cannot enter the basis in the next iteration. 


[3.36] Prove or give a counterexample. In order to have a basic variable in a 
particular row of the simplex tableau, that variable must have a nonzero 
coefficient in its original column and the particular row. 


[3.37] Consider the linear programming problem: Maximize ex subject to Ax = 
b, x > 0, where A is an m X n matrix of rank m. Suppose that an optimal solution 
with basis B is at hand. Further suppose that b is replaced by b + Ad where A is a 
scalar and d is a fixed nonzero vector of dimension m. Give a condition such 
that the basis B will be optimal for all A > 0. 

[3.38] Write a precise argument showing that in the absence of degeneracy and 
assuming feasibility, the simplex method will provide an optimal solution or 
reveal unboundedness of a linear program in a finite number of steps. 
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[3.39] Suppose that some tableau for a linear programming problem exhibits an 
indication of unboundedness. Considering only the basic vectors and the non- 
basic vector corresponding to the column giving the unboundedness indication, 
demonstrate which entities, if any, satisfy the definition of an extreme point. 
Also demonstrate which entities, if any, satisfy the definition of an extreme ray. 
Give an example. 


[3.40] Consider the linear program: Minimize ex subject to Ax > b, x > 0. 
Converting the inequality constraints to equality constraints, suppose that the 
optimal basis is B. Show that w= ¢ ,B! 2 0. 


[3.41] Suppose that we know a priori that a solution cannot be optimal unless it 
involves a particular variable at a positive value. Show that this variable can be 
eliminated and that the reduced system with one less equation and variable can 
be solved in its place. Illustrate by an example. 


[3.42] Consider the problem to minimize z = ex, subject to Ax < b and x > 0. 
Hence, we are to find the smallest z for which the inequality system z > ex, Ax < 
b, x = 0 has a solution. Consider the variable x,. Using each inequality with x, 
on one side and all other terms on the other side, show how x, can be 


eliminated from the problem. Hence, show how this can be repeated to solve the 
linear program. Illustrate using the example in Exercise 3.7. (This is known as 
the Fourier~Motzkin Elimination Method.) 


[3.43] a. Characterize the set of alternative optimal solutions given the following 
optimal tableau: 


x X x3 x4 x5 X6 RHS 


b. Consider the problem to maximize ex subject to the inequality constraints 
depicted below. Give all bases for which z; —c, 2 0 for the nonbasic variables. 


x24 
c 
te " 
[3.44] Consider the following linear program: 
Minimize -—x,} — 2x + x3 
subjectto 2x, + 2% + »% S 6 
2X5 -— xX < 3 
X15 Xp, x, 2 0. 
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a. Find an optimal solution by the simplex method. At each iteration 
identify B, N, Bo, BN, cpB, and the (S55 c; )—values. 

b. At optimality, find Ox, /Ox3, Ox /Ox4, 0z/Oxs5, OXp/Oxy, Where x4 
and xs are the slack variables. Interpret these values. 

c. Suppose that c, is changed from —] to—-1+ Aj, and c is changed 
from —2 to -—2 + A,. Find the region in the (A,, Ay) space that will 


maintain optimality of the solution you obtained in Part (a). 
d. Suppose that a new activity xg is considered. Here cg =—4, ayg = 2, 


and ay, = 4. Is it worthwhile to produce the activity? If your answer 


is yes, find the new optimal solution. 
e. Suppose that b; is changed from 6 to 6 + A. Find the range of A that 


will maintain optimality of the basis found in Part (a). 
f. Make use of the final tableau to represent the column a3 as a linear 


combination of a; and ap. 


[3.45] Consider the following linear programming problem: 


Maximize 2x, + 12x. + 7x3 


IA 


subjectto xy + 3x, + 2x3 < 10000 
2x; + 2X + X3 < 4000 
0. 


IV 


X1> x2, x3 


The optimal solution is shown below, where z is the objective function and x4 
and xs are slack variables: 


Zz x, x x X4 Xs RHS 


z 7 | 28000 
x4 -2 | 2000 
x 1 | 4000 


a. What are the rates of increase of the objective as a function of the 
right—hand-side of the first and second constraints, respectively? 

b. Suppose that the right—hand-side of the second constraint is changed 
to 4000 + A. What is the range of A that will keep the basis of the 
foregoing tableau optimal? 

c. Find explicitly the optimal value z as a function of A for Part (b). 

d. Suppose that increasing the right-hand-side of the second constraint 
involved expanding a manufacturing department. This will involve a 
fixed cost as well as a variable cost that is a function of A. In 
particular, the cost as a function of A is 


0 if A = 0 


h{A) = 
4000 + 2A if A> 0. 
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What is the break-even point where the cost and the added profit will 
balance? What do you recommend for the optimal value of A? 
[3.46] Consider the following simplex tableau for a minimization problem (the 
constraints are of the < type and x3, x4, and x5 are the slacks). 


Suppose that a <0, b <0, and c, d, e>0. 
a. FindB™!, 
Find B. 


b 
c. Is the tableau optimal? 

d. Give the original tableau (in terms of the unknowns). 
e 


From the tableau identify c Re and give its interpretation. 


Now, suppose that a > 0, b < 0; and c, d, e>0. 

f. Is the new tableau optimal? 

g. Give an extreme direction. 

h. Leta=5 and f=~-10. Give a feasible solution having z = -150. 
[3.47] The following is the current simplex tableau of a linear programming 
problem. The objective is to minimize —2xg — x5 — 2x¢, and x, x2, and x3 are 
the slack variables. 


Find the values of the unknowns a through A in the tableau. 

Find B™. 

Find Ox2 /0x,, 02/ 0x5, Ox¢ / Ob3. 

Without explicitly finding the basic vectors ag, a7, a4, give the rep- 


oo fF 


resentation of the vector a5 in terms of these basic vectors. 


[3.48] The starting and current tableaux of a given problem are shown below. 
Find the values of the unknowns a through n. 
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Starting tableau 


Current tableau 2/3 2/3 1/3 0 
i -Ip 1 


[3.49] The following is the current simplex tableau of a given maximization 
problem. The objective is to maximize 2x, — 4x>, and the slack variables are x; 


and x4. The constraints are of the < type. 


a. Find the unknowns a through g. 
b. Find B'. 
c. Find 0x3/0x,, 0z/0b,, Oz/ 0x4, Ox, / Ob. 
d. Is the tableau optimal? 
[3.50] Consider the problem: Maximize ex subject to Ax = b, x unrestricted in 
sign. Under what conditions does this problem have a bounded optimal 
solution? 
[3.51] Consider a linear programming problem in which some of the variables 
are unrestricted in sign. What are the conditions for a bounded optimal solution? 
Without introducing additional variables, show how the entry and exit criteria of 
the simplex method can be modified such that the unrestricted variables are 


handled directly. How does the simplex method recognize reaching an optimal 
solution in this case? Illustrate by solving the following problem. 


Minimize -3x, + X) 


subjectto 2x, + 3x, < 6 
xX 7 x2 < 6 
x, 2 0 

Xx) unrestricted. 


[3.52] A necessary and sufficient condition for unboundedness of the objective 
value of a (feasible) minimization problem is that there exists a direction of the 
feasible region such that ed < 0. A condition for unboundedness in the simplex 
method is the existence of an index j such that z 7 7 ¢; 2 0and y; <0. Discuss 


in detail the correspondence between the two conditions. 
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(Hint. Let 


where the | appears at the jth position. Show that d is a direction of the set and 
that ed = c; — z;, Can you show that d is an extreme direction?) 


[3.53] Construct a detailed flow diagram of the simplex method. What are the 
number of operations (additions, subtractions, multiplications, divisions) that are 
needed at each simplex iteration? 

[3.54] Consider the linear program to maximize cx subject to Ax = b and x > 0, 
where A is m x n of rank m. Let B be any basis matrix, and let B* be the basis 
associated with any optimal basic feasible solution. Define z; = ¢ ,B! a; and 


* 


zj =e Br a, for all j. Consider a particular variable x; for which there 


exists a set of m columns ajc j)> j = 1,...,m, chosen from [A, b], and a 


corresponding set of m scalars Gj jo J=1,...m, such that 


~| m 
Bac mse Meaty. S 9 
j=l 
and 
m * 
(2; - ci) - 2 Glia ~ 2] > 0. 
j= 


Then show that x; 


; must be nonbasic at optimality. Is this generally 


implementable? Can you identify an implementable special case? 
(Hint: Since c ,B’'B 2 Cp (why?), obtain an inequality by postmultiplying 


both sides of this by the expression in the first condition and then use the second 
condition.) 


NOTES AND REFERENCES 


1. This chapter describes the simplex algorithm of Dantzig (developed in 
1947 and published at a later date in 1949). The material of this chapter is 
standard and can be found in most linear programming books. The histori- 
cal information is from Dantzig [1982]. 


2h In Section 3.1 we proved optimality at an extreme point via the 
Representation Theorem. The reader may note that the simplex algorithm 
itself gives a constructive proof of this optimality result. 
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3. The material on block pivoting is from Tucker [1960b]. For further read- 
ing on block pivoting, see Dantzig [1963a], Lasdon [1970], and Cooper 
and Kennington [1979]. For a discussion on some nonadjacent extreme 
point methods for linear programs, see Sherali, Soyster, and Baines 
[1983], Sherali and Baines [1984], Murty and Fathi [1984], Murty 
[1986], and Sethi and Thompson [1984]. A host of nonlinear program- 
ming approaches to linear programming problems have been inspired by 
Khachian’s [1979] and Karmarkar’s [1984a, b] polynomial-time algo- 
rithms for linear programs (see Chapter 8). 

4. Exercise 3.54 relates to Cheng’s [1980] theoretical work (see also 
Brosius [1981]) on a priori recognizing basic or nonbasic variables at 
optimality. 
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FOUR: STARTING SOLUTION 
AND CONVERGENCE 


In the previous chapter, we developed the simplex method with the assumption 
that an initial basic feasible solution is at hand. In many cases, such a solution is 
not readily available, and some work may be needed to get the simplex method 
started. In this chapter, we describe two procedures (the two-phase method and 
the big—M method), both involving artificial variables, to obtain an initial basic 
feasible solution to a slightly modified set of constraints. The simplex method is 
used to eliminate the artificial variables and to then solve the original problem. 
We also discuss in more detail the difficulties associated with degeneracy. In 
particular we show that the simplex method converges in a finite number of 
steps, even in the presence of degeneracy, provided that a special rule for 
entering and/or exiting from the basis is adopted. 


4.1 THE INITIAL BASIC FEASIBLE SOLUTION 


Recall that the simplex method starts with a basic feasible solution and moves to 
an improved basic feasible solution, until the optimal point is reached or 
unboundedness of the objective function is verified. However, in order to 
initialize the simplex method, a basis B with b = B 'b>0 must be available. 
We shall show that the simplex method can always be initiated with a very 
simple basis, namely, the identity. 


Easy Case 

Suppose that the constraints are of the form Ax < b, x > 0 where A is an m xn 
matrix and b is a nonnegative m—vector. By adding the slack vector x,, the 
constraints can be put in the following standard form: Ax + x, =b,x 20, x, > 
0. Note that the new m x (m+n) constraint matrix (A, I) has rank m, and a basic 
feasible solution of this system is at hand, by letting x, be the basic vector, and 
x be the nonbasic vector. Hence, at this starting basic feasible solution, we have 
x, =b and x = 0, and now the simplex method can be applied. 


Some Bad Cases 


In many situations, finding a starting basic feasible solution is not as straight- 
forward as the case just described. To illustrate, suppose that the constraints are 
of the form Ax < b, x > 0, but the vector b is not nonnegative. In this case, after 
introducing the slack vector x,, we cannot let x = 0, because x, = b violates the 
nonnegativity requirement. 

Another situation occurs when the constraints are of the form Ax > b, x > 
0, where b < 0. After subtracting the slack vector x,, we get Ax — x,= b, x > 
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0, and x, > 0. Again, there is no obvious way of picking a basis B from the 


matrix (A, —I) with b = B7! b>0. 
In general any linear programming problem can be transformed into a 
problem of the following form: 


Minimize cx 
subject to Ax=b 
x>0, 


where b > 0 (if 5; < 0, the ith row can be multiplied by —1). This can be 
accomplished by introducing slack variables and by simple manipulations of the 
constraints and variables (see Chapter 1 for details). If A contains an identity 
matrix, then an immediate basic feasible solution is at hand, by simply letting B 
= I, and since b > 0, then B!b=b2>0. Otherwise, something else must be 
done. A trial-and-error approach may be futile, particularly if the problem is 
infeasible! 


Example 4.1 


a. Consider the following constraints: 


x + 2X < 4 
—X} + Xo <1 
X15 x4 2 0. 


After adding the slack variables x3 and x4, we get 


xy + 2X7 + X3 = 4 
xX] + x2 + X4 = 1 
X15 x2 5 x3 5 X4 > 0. 


An obvious starting basic feasible solution is given by 


EG) = s-)-(h 


b. Consider the following constraints: 


X + X + X% < 6 
—2x,; + 3x, + 2x, 2 3 
xX, x3 2 0 


Note that x, is unrestricted. So the change of variable x, = xf — x, 
is made. Also, the slack variables x4, and x5 are introduced. This 
leads to the following constraints in standard form: 


xp o- x + xX + X + % = 6 
-2xf + 2x7 + 3x, + 2x3 — Xs, = 

+ — 

xX} 5 xX} 5 x2, X35 X45 Xs > 0. 


Ww 
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Note that the constraint matrix does not contain an identity and no 
obvious feasible basis B can be extracted. 
c. Consider the following constraints: 


X + X% —- 2x, < -3 
-2x, + X, + 3x3 < 7 
x, Xp, x3 2 0. 


Since the right-hand-side of the first constraint is negative, the first 
inequality is multiplied by —1. Introducing the slack variables x4 and 


xs leads to the following system: 


—X% -— X. + 2x3 - X% = 3 
—2x, + xX + 3x3 + X¥5 = 7 
Xy, X>, x3, X4; x5 20 


Note again that this constraint matrix contains no identity. 


Artificial Variables 


After manipulating the constraints and introducing slack variables, suppose that 
the constraints are put in the format Ax = b, x > 0 where A is an m x n matrix 
and b > 0 is an m—vector. Furthermore, suppose that A has no identity submatrix 
(if A has an identity submatrix then we have an obvious starting basic feasible 
solution). In this case, we shall resort to artificial variables to get a starting basic 
feasible solution, and then use the simplex method itself and get rid of these 
artificial variables. 

To illustrate, suppose that we change the restrictions by adding an 
artificial vector x, leading to the system Ax + x, =b,x>0, x, > 0. Note that 


by construction, we created an identity matrix corresponding to the artificial 
vector. This gives an immediate basic feasible solution of the new system, 
namely, x, = b and x = 0. Even though we now have a starting basic feasible 


solution and the simplex method can be applied, we have in effect changed the 
problem. In order to get back to our original problem, we must force these 
artificial variables to zero, because Ax = b if and only if Ax + x, =b withx, = 


0. In other words, artificial variables are only a tool to get the simplex method 
started; however, we must guarantee that these variables will eventually drop to 
zero, if at all possible. 

At this stage, it is worthwhile to note the difference between slack and 
artificial variables. A slack variable is introduced to put the problem in equality 
form, and the slack variable can very well be positive, which means that the 
inequality holds as a strict inequality. Artificial variables, however, are not 
legitimate variables, and they are merely introduced to facilitate the initiation of 
the simplex method. These artificial variables, however, must eventually drop to 
zero in order to attain feasibility in the original problem. 


Example 4.2 


Consider the following constraints: 
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xX) + 2x 2 4 
—3x, 4x5 > 5 
2x) +t XxX) < 6 

xX; xX, 2 0. 


Introducing the slack variables x3, x4, and x5, we get 


xX + 2x, - x3 4 
—3x, + 4x — X4 5 
2X) + Xo + XS = 6 

xX, X9, x3, X4, X5 2 0. 


This constraint matrix has no identity submatrix. We can introduce three artifi- 
cial variables to obtain a starting basic feasible solution. Note, however, that Xs 
appears in the last row only and it has a coefficient of 1. So we only need to 
introduce two artificial variables xg and x7, which leads to the following system: 


Legitimate variables Artificial variables 
xX + 2X -— %3 + XG = 4 
-3x, + 4x -— x4 + x7 = § 
2x, + XX + Xs = 6 
xy, Xp, x3, X4, x5, X6> xy 2 0. 


Now, we have an immediate starting basic feasible solution for the new system, 
namely, x5 = 6, x, =4, and x, = 5. The rest of the variables are nonbasic and 


have a value of zero. Needless to say, we eventually would like the artificial 
variables xg and x7 to drop to zero. 


4.2 THE TWO-PHASE METHOD 


There are various methods that can be used to eliminate the artificial variables. 
One of these methods is to minimize the sum of the artificial variables, subject 
to the constraints Ax + x, =b,x >0and x, > 0. Ifthe original problem has a 


feasible solution, then the optimal value of this problem is zero, where all the 
artificial variables drop to zero. More importantly, as the artificial variables drop 
to zero, they leave the basis, and legitimate variables enter instead. Eventually, 
all artificial variables leave the basis (this is not always the case, because we 
may have an artificial variable in the basis at level zero; this will be discussed 
later in greater detail). The basis then consists of legitimate variables. In other 
words, we get a basic feasible solution for the original system Ax = b, x > 0, and 
the simplex method can be started with the original objective function ex. If, on 
the other hand, after solving this problem we have a positive artificial variable, 
then the original problem has no feasible solution (why?). This procedure is 
called the two-phase method. In the first phase, we reduce artificial variables to 
value zero or conclude that the original problem has no feasible solutions. In the 
former case, the second phase minimizes the original objective function starting 
with the basic feasible solution obtained at the end of Phase I. The two-phase 
method is outlined below: 
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Phase I 


Solve the following linear program starting with the basic feasible solution x = 0 
and x, = b: 


Minimize xX =x, 
subject to Ax+ x, =b 
x, x, 20. 


If at optimality x, # 0, then stop; the original problem has no feasible solutions. 
Otherwise, let the basic and nonbasic legitimate variables be xp and xy. (We 


are assuming that all artificial variables left the basis. The case where some 
artificial variables remain in the basis at zero level will be discussed later.) 
Proceed to Phase II. 


Phase II 
Solve the following linear program starting with the basic feasible solution xp = 


B | band xy =0: 


Minimize Z= Cp Xp + Cy Xy 
subject to Xp t+ BINxy =B!b 
XB,Xy >0. 


The foregoing problem is of course equivalent to the original problem. 


Example 4.3 
Minimize x — 2x, 
subjectto x + XxX 2 2 
—xy + XxX 2 1 
X49 < 3 
xX, X94 = 0 


The feasible region and the path taken by Phase I and Phase IJ to reach the 
optimal point are shown in Figure 4.1. After introducing the slack variables x3, 


Xq4, and x5, the following problem is obtained: 


Minimize x, — 2x 
subject to x + XX - % = 2 
—Xy + XQ — X4 =] 
xX + x5 = 3 
x, Xp, x3, X4, x, 2 0. 


An initial identity is not available. Hence, we introduce the artificial variables 
X— and x7 (note that the last constraint does not need an artificial variable). 


Phase I starts by minimizing xq = x6 + x7. 
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lo x} 
¢ 


Figure 4.1. Example of the two—phase method. 


Phase I 


ARTIFICIALS 


=/2 ‘1/2 jr 12 
Ayo: 1/2 2 1/2 
y2 1/2 si <12 
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This is the end of Phase I. We have a starting basic feasible solution, (44, 
X7) = (1/2, 3/2). Now we are ready to start Phase II, where the original objec- 
tive is minimized starting from the extreme point (1/2, 3/2) (see Figure 4.1). 
The artificial variables are disregarded from any further consideration. 


Phase II 


Zy— Cy = 2 — cy = 0. 


Since z; — c; <0 for all nonbasic variables, the optimal point (0, 3) with objec- 


tive value —6 is reached. Note that Phase [ moved from the infeasible point (0, 
0), to the infeasible point (0, 1), and finally to the feasible point (1/2,3/2). 
From this extreme point, Phase II moved to the feasible point (0, 2) and finally 
to the optimal point (0, 3). This is illustrated in Figure 4.1. The purpose of Phase 
I is to get us to an extreme point of the feasible region, while Phase II takes us 
from this feasible point to an optimal point, assuming that an optimum exists. 


Analysis of the Two—Phase Method 


At the end of Phase I, either x, # 0 or else x, = 0. These two cases are discussed 
in detail below: 
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Case A: x, #0 

If x, #0, then the original problem has no feasible solution, because if there is 
an x > 0 with Ax = b, then (°) is a feasible solution of the Phase I problem and 
0(x) + 100) = 0 < 1x,, violating optimality of x,. 

Example 4.4 

(Empty Feasible Region) 


Minimize —3x, + 4x, 


subjectto x + X% < 4 
2x, + 3x, 2 18 
x, xX, => 0. 


The constraints admit no feasible solution, as shown in Figure 4.2. This will be 
detected by Phase I. Introducing the slack variables x3; and x4, we get the 


following constraints in standard form: 


X +t XX + X3 = 4 
2x, + 3x, — x4 = 18 
X15 X25 X35 X4 2 0. 


Since no convenient basis exists, we introduce the artificial variable x5 into the 
second constraint. Phase I is now used to try to get rid of the artificial variable. 


Phase I 


x 


Figure 4.2. Empty feasible region. 
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act) 
x3 
x5 


x0 
x2 
x5 


The optimality criterion of the simplex method, namely z; — c; < 0, holds for 


all variables; but the artificial variable x5 > 0. We conclude that the original 
problem has no feasible solutions. 


Case B: x, =0 


This case is further divided into two subcases. In subcase BI, all the artificial 
variables are out of the basis at the end of Phase I. The subcase B2 corresponds 
to the presence of at least one artificial variable in the basis at zero level. These 
cases are discussed in turn below: 


Subcase B1 (All Artificials Are Out of the Basis) 


Since at the end of Phase I we have a basic feasible solution, and since x, is 


out of the basis, then the basis consists entirely of legitimate variables. If the 
legitimate vector x is decomposed accordingly into xg and xj, then at the end 


of Phase I, we have the following tableau: 


Now, Phase II can be started with the original objective function, after 
discarding the columns corresponding to x,. (For academic purposes, these 


columns may be kept since they would present B! at each iteration. Note, 
however, that an artificial variable should never be permitted to enter the basis 
again.) The (z; — c;)-values for the nonbasic variables are given by the vector 


c 2B 'N — ¢,,, which can be easily calculated from the matrix BN stored in 
the final tableau of Phase I. The following initial tableau of Phase II is 
constructed. Starting with this tableau, the simplex method is used to find an 
optimal solution or to detect unboundedness. 
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Subcase B2 (Some Artificials Are in the Basis at Zero Values) 


In this case we may proceed directly to Phase II, or else eliminate the artificial 
basic variables and then proceed to Phase II. These two actions are discussed in 
further detail next. 


PROCEED DIRECTLY TO PHASE II 


First, we eliminate the columns corresponding to the nonbasic artificial vari—- 
ables of Phase I. The starting tableau of Phase JI consists of some legitimate 
variables and some artificial variables at zero values. The cost row, consisting of 
the (z; — c;)-coefficients, is constructed for the original objective function so 


that all legitimate variables that are basic have z; —-c; = 0. The cost 


coefficients of the artificial variables are given value zero (justify!). While 
solving the Phase II problem by the simplex method, we must be careful that 
artificial variables never reach a positive level (since this would destroy 
feasibility). To illustrate, consider the following tableau, where for simplicity 
we assume that the basis consists of the legitimate variables x,, x»,...,x, and 


the artificial variables x,,.441, -..X,+m (the artificial variabies x,.1,....X,+, left 
the basis during Phase I): 


Suppose that z; — c; > 0, and so x; is eligible to enter the basis, where 


x; 1s a legitimate nonbasic variable. If vy 2 0 fori=k+ 1,..., m, then the 


j 
artificial variable x,,; will remain zero as x; enters the basis if y,; = 0, and 


otherwise, if Vy > 0, then x j will enter the basis at the zero level. In either 


case, the usual minimum ratio test can be performed, preferring to exit an artifi- 
cial variable from the basis in the case of ties. If, on the other hand, at least one 
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component y,, <0,r e€ {k + 1, ..., m}, then the artificial variable x,,, will 


become positive as x; is increased. This action must be prohibited since it 


would destroy feasibility. This can be done by pivoting at y,; rather than using 
the usual minimum ratio test. Even though Vy <9, pivoting at Vy would main- 


tain feasibility since the right-hand-side at the corresponding row is zero. In 
this case, x; enters the basis and the artificial variable x,,, leaves the basis, 


and the objective value remains constant. In all cases, nonbasic artificial 
variable columns are deleted from the tableau. With this slight modification the 
simplex method can be used to solve Phase II. 


FIRST ELIMINATE THE BASIC ARTIFICIAL VARIABLES AT THE 
END OF PHASE I 


Rather than adopting the preceding rule, which nonetheless guarantees that 
artificial variables will always be zero during Phase II, we can eliminate the 
artificial variables altogether before proceeding to Phase II. The following is a 
typical tableau (possibly after rearranging) at the end of Phase I. The objective 
row and column do not play any role in the subsequent analysis and are hence 
omitted. 


BASIC NONBASIC NONBASIC BASIC 
LEGITIMATE LEGITIMATE ARTIFICIAL ARTIFICIAL 
VARIABLES VARIABLES VARIABLES VARIABLES 


We now attempt to drive the artificial variables x,,4;41,..5X,4m Out of 
the basis by placing m — & of the nonbasic legitimate variables x;,4),...,X, into 
the basis, if possible. For instance, x,,,;4,,; can be driven out of the basis by 
pivoting at any nonzero element of the first row of Rz. The corresponding 
nonbasic legitimate variable enters, x,,,,4, leaves the basis, and the tableau is 


updated. This process is continued. If all artificial variables drop from the basis, 
then the basis would consist entirely of legitimate columns, and we proceed to 
Phase II as described in subcase BI. If, on the other hand, we reach a tableau 
where the matrix R» = 0, none of the artificial variables can leave the basis by 


introducing x; 4), OF X;475---5 OF X,. Denoting (x,,x7,-...x%)° and (%p4y,-5%,)! 
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A A 
by x, and x5, respectively, and decomposing A and b accordingly into a] 
21 22 


b 
and K | it is clear that the system 
2 


k n-k 
kA} An | Ara Px | | By 
m—k| Ao, | Aa? || Xo b, 
is transformed into 
k n-k 


k [ER [x] _ fb, 

m-k\0| 0 |lx,]| | 0 
through a sequence of elementary matrix operations. This shows that rank (A, b) 
= k < m; that is, the last m — & equations are algebraically redundant and 


R, = Ay Aj, and b; = Aj\b;. The last m — k rows of the last tableau can be 


thrown away, and we can proceed to Phase II without the artificial variables. 
The basic variables are x,, > ,...,x,; and the nonbasic variables are x; 41 ,...,X;)- 


The starting tableau of Phase II is depicted below, where eg = (cy, C2...-, Cy): 


Zz Xp XE Keer hy RHS 


Example 4.5 
(Redundancy) 


Minimize -x, + 2x 


subjectto x + Xx, + 3 = 6 
Xx, + x2 + 2x3 = 4 

2x> + 3x3 = 10 

X3 < 2 

M, Xp, x3 2 0. 


After introducing a slack variable, x4, in the fourth constraint, the constraint 
matrix A is given by: 
1 1 1 £0 
_}-!} 12 0 
sosiicdl Re ee ee et 
00 1 1 


Note that the matrix is not of full rank, since if we add the first two rows of A, 
we get the third row; that is, any one of the first three constraints is redundant 
and can be thrown away. We shall proceed as if this fact were not known, 
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however, and introduce the artificial variables x5, xg, and x7. The Phase I 


objective is: Minimize xg = xs + x6 + x7. Phase I proceeds as follows: 


Phase I 


1/2 
3/2 


1/2 
1/2 


-1/2 
xX» 1/2 
7 


x3 


Since all the artificial variables are at a level zero, we may proceed to 
Phase II with a basic feasible solution to the original problem at hand. We can 
either proceed directly with the artificial x7 in the basis at the zero level or 


attempt to eliminate x7 from the basis. The only legitimate nonbasic variable is 
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x4, and it has zero coefficient in row 3 corresponding to x7. This shows that the 


third row (constraint of the original problem) is redundant and can be thrown 
away. This will be done as we move to Phase II. (Indeed, observe that by setting 
the artificial variables x5, x6, and x7 to zero, this third equation row reads 


simply as 0 =0.) 
PHASE II 


Obviously, z; — ¢, = 22 — Co = 23 —c3 = 0. Also, x5 and xg are nonbasic 
artificial variables and will be eliminated from the Phase II problem. In order to 
complete row 0 we need to calculate z4 — c4: 


2747-4 = cpB a, — 


1/2 
(1,253) 3/2) <0 
1 


—13/2. 


Since we are minimizing and z4 — c4 < 0 for the only nonbasic variable, then 


we stop; the solution obtained from Phase I is optimal. The following tableau 
displays the optimal solution: 


Zz xy x2 x3 X4 RHS 


Organization of the Tableau in the Two-Phase Method 


To eliminate the need for recomputing the (z j 7 ¢;)-values when the Phase I 


objective function is replaced by the Phase II (original) objective function, an 
extra row could be added to the tableau representing the original cost 
coefficients. The following represents the setup of the initial tableau (not yet in 
canonical form): 


ARTIFICIALS 
Zz X9 x “Xp Xn4t tt Xntm _ RHS 
2 [JO [aq = gO OT OJ Phase Objective 


To convert this tableau to canonical form for Phase I (that is, unit vectors for all 
basic variables), we must perform preliminary pivoting to obtain zeros for x,41 
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through x, in the x9 (Phase J objective) row. This is done by successively 
adding every row (except the z row) to the xg row. 


Once the initial tableau has been converted to canonical form, the 
simplex method is applied to the resulting tableau using the x9 row as the 
(Phase I) objective function row until optimality is achieved. During Phase I the 
z row is transformed, just as any other row of the tableau would be, to maintain 
unit vectors for the basic variables. Note, however, that during Phase J, although 
the basis entry is solely determined by the entries in row xg, ties could be 


broken using the z row. 
At the end of Phase I, if xp is positive, then the problem is infeasible and 


the optimization process can be terminated. Otherwise, the x9 row and xg 


column are deleted and Phase II is initiated (after possibly eliminating artificial 


variables), with the values in the z row being the correct values of z pees 


(why?) for the Phase II objective. 


4.3 THE BIG-M METHOD 


Recall that artificial variables constitute a tool that can be used to initiate the 
simplex method. However, the presence of artificial variables at a positive level, 
by construction, means that the current point is an infeasible solution of the 
original system. The two-phase method is one way to dispense with the 
artificial variables. However, during Phase I of the two-phase method the 
original cost coefficients are essentially ignored. Phase I of the two-phase 
method seeks any basic feasible solution, not necessarily a good one. Another 
possibility for eliminating the artificial variables is to assign coefficients for 
these variables in the original objective function in such away as to make their 
presence in the basis at a positive level very unattractive from the objective 
function point of view. To illustrate, suppose that we want to solve the 
following linear programming problem, where b > 0: 


Minimize ¢x 
subjectto Ax = Db 
x >= 0. 
If no convenient basis is known, we can introduce the artificial vector x,, 
which leads to the following system: 


Ax+X, 


b 
X,Xq 0. 


lV Il 


The starting basic feasible solution is given by x, = b and x = 0. In order to 


reflect the undesirability of a nonzero artificial vector, the objective function is 
modified such that a large penalty is paid for any such solution. More specifi- 
cally consider the following problem: 
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Minimize Zpjp.y =cx + MI1x, 
b 
0, 


subject to Ax + x 
x, x 


a 


MoI 


a 
where M is a very large positive number (see Section 4.4). The term M1x, can 
be interpreted as a penalty to be paid by any solution with x, 4 0. Alternatively, 
the foregoing strategy can be interpreted as one that minimizes 1x, with priority 


one, and among all alternative optimal solutions for this objective, minimizes 
the secondary objective ex. Hence, even though the starting solution x = 0, x, = 


b is feasible to the new constraints, it has a very unattractive objective value, 
namely Mlb. Therefore, the simplex method itself will try to get the artificial 
variables out of the basis, and then continue to find an optimal] solution to the 
original problem, if one exists. We call this technique the big-M method. 

The big—M method is illustrated by the following numerical example. 
Validation of the method and possible cases that might arise are discussed 
subsequently. 


Example 4.6 
Minimize z=x, — 2x 
subject to Xo + xX 22 
—Xy) + XQ > 1 
x4 < 3 
xX}, x4 > 0. 


This example was solved earlier by the two—phase method (Example 4.3). The 
slack variables x3, x4, and x5 are introduced and the artificial variables x¢ and 


X7 are incorporated in the first two constraints. The modified objective function 
IS Zpig-M = % ~ 2X2 + Mxg + Mx, where Mis a large positive number. This 


leads to the following sequence of tableaux: 


2big -M x x %3 x4 x5 XG x7 RHS 
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1/2 
0 1/2 1/2 
1/2 ae 1/9 3/2 


Since z; — c,; <0 for each nonbasic variable, the last tableau gives an optimal 


solution. The sequence of points generated in the (x,, x2) space is illustrated in 
Figure 4.3. 


Analysis of the Big-M Method 
We now discuss in more detail the possible cases that may arise while solving 


the big—M problem. The original problem P and the big—M problem P(M) are 
stated below, where the vector b is nonnegative: 


~ x 
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Figure 4.3. Example of the big—M method. 


Problem P: Minimize z=cx 
subject to Ax = b 
x > 0 


Problem P(M): Minimize Zbig-M =cx + M1x, 
subject to Ax + x, =b 
X, x, 20. 


Since Problem P(M) has a feasible solution (e.g., x = 0 and x, = b), then while 
solving it by the simplex method one of the following two cases may arise: 


1. We arrive at an optimal solution of P(M). 
2. We conclude that P(M) has an unbounded optimal objective value, 
that is, z > —oo. 


Of course, we are interested in conclusions about Problem P and not 
P(M). The following analysis will help us to draw such conclusions. 

The key observation that makes this analysis simple and transparent is 
that the objective function of the big—M method is simply the sum of the Phase 
II objective and M times the Phase I objective of the two-phase method. In 
other words, noting the objective function of Problem P(M) and that of the 
Phase I problem in Section 4.2, we have that 


Zbig-M =Zt+ Mx. 


In fact, for any basic feasible solution to the artificial problem, the 
canonical form of the objective row in the big—M method is precisely the sum of 
the canonical form of the Phase II objective and M times the canonical form of 
the Phase I objective in the two-phase method. Indeed, each of these objective 
functions has simply been rewritten in terms of the nonbasic variables. For 
example, examine the sequence of tableaux produced for Example 4.3 using the 
two-phase method with those produced for this same problem in Example 4.6 
using the big—M method. Notice that the coefficients of M in the objective row 
Zpig-m for the tableaux in Example 4.6 are precisely the objective coefficients 


in the Phase I objective row Xp for the tableaux in Example 4.3, up to the point 


when the artificial variables are all driven out of the basis. At this point, when 
the two-phase method switches to Phase II, the objective coefficients in the z 
row reveal the constants not associated with M in the Zpj9..4 representation 


(where the nonbasic artificial variables have been eliminated). Consequently, 
the two—phase method avoids possible round—off computational error problems 
associated with a large value of M by carrying the coefficients of M in the big— 
M method separately as Phase I objective coefficients, and by carrying the other 
coefficients as Phase II objective coefficients. 
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Hence, for example, the Zpig.4y objective row for the tableau at the 


second iteration of Example 4.6 can be decomposed into its components z and 
xg using the preceding relationship, Zpjg.44 = Zz + Mxg, as follows: 


Notice that our first priority is to minimize x). Viewing the objective row for 


xq (equivalently, viewing the coefficients of M in Zpig.y), we notice that 


Phase I is not complete. Hence, we enter x, and continue as in Example 4.6. 


The rules for operating the big-M/ method are now directly evident by 
thinking of this process as applying the two-phase method, while carrying the 
canonical representation of both the original objective function z and the artifi- 
cial objective function x9. While the coefficients of M in the Zpjy.4y Tow are 


not all nonpositive (< 0), i.e., so long as some z; — c; in the Zpjg.4¢ row has a 


positive coefficient for M, we are not done with Phase I, and we select such a 
nonbasic variable to enter the basis. Notice that this will automatically occur if 
we select the nonbasic variable having the largest (z; — c;)-value to enter the 


basis. Once all the coefficients of M in the (z; — c;)-values in the Zpjg.4y Tow 


are nonpositive, we know that Phase I is done. At this point, we pause and view 
the values of the artificial variables. The following two cases can arise: 


Case A: The artificial variables are all equal to zero 


In this case, the original problem is feasible, and in fact, we have a basic feasible 
solution to this problem. (The artificial variables can be eliminated from the 
problem, including the steps of having to possibly pivot out degenerate artificial 
variables from the basis before deleting them, or deleting redundant rows, as 
discussed for the two-phase method.) We can now continue with the simplex 
method as usual, in effect applying it to the original problem itself, until 
optimality or unboundedness is detected. 


Case B: Some artificial variable is positive 


In this case, as with the two-phase method, we can conclude that the original 
problem is infeasible (because Phase I is over and some artificial variable is still 
at a positive level). Note that in this case, we can stop the optimization process, 
even though the Zpig.,¢ row might still have some z; — c; > 0 and even perhaps 


LO 9 
reveal that Problem P(M) is unbounded. 
The following examples illustrate these concepts: 
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Example 4.7 
(No Feasible Solutions) 


Minimize —x, — 3x. + 3 
subjectto x, + XxX, + 2x3 < 4 
—xy + 3 24 
x3 2 3 
X, Xp, x3 2 0. 


Introduce the slack variables x4, x5, and xg. Also, introduce the two artificial 
variables xz and xg for the last two constraints with cost coefficients equal to 
M. This leads to the following sequence of tableaux: 


Zrig-M DHS HCH 


Multiply rows 2 and 3 by M and add to row 0. 


Zbig-M 2 3% SH X7_ gp -RHS 
3 -1+2M 0 -M -M 


2big-M x x %3 x4 % % % % RHS 
Zbig- M 3/2 - 2M 0 
X3 1/2 0 
x7 —3/2 -1/2 Oo -1/2 -l 0 
Xg -1/2 1 


Since the coefficients of M in the Zpj9.,4 row are all nonpositive, Phase I is 


ay gree a 
jos 


0 for all nonbasic variables; that is, the tableau is optimal.) However, the 
artificial variables x7 and xg still remain in the basis at a positive level, and so 


we conclude that the original problem has no feasible solutions. 


complete. (In fact, in this example, because M > 0 is very large, then z 


Example 4.8 
Minimize -x; — Xx, 
subject to x - x -— x% = 1 
xX) + X% + 2x3 - xy = 1 
xX, Xp, x3, x4 2 0 
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Introduce the artificial variables x; and xg with cost coefficient M. This leads 
to the following sequence of tableaux: 


=(/2)M_0 


=(1/2)M 


Xs -1/2 Or 2. 1/2 3/2 
x3 1/2 1 -/2 0 1/2 2 
Zbig-M M1 XD RONG Xs X6 RHS 
Zbig-M 
x 
%3 


Notice that the coefficients of M in the Zpig_y4 row are all nonpositive. Hence, 


Phase I is complete. Furthermore, all the artificial variables are zero (and in fact 
nonbasic). Hence, the original problem is feasible. Eliminating (or ignoring) the 


artificial variables, we see now that the most positive z 77 corresponds to 


X_ and that yy <0. Therefore, the original program has an unbounded optimal 
value along the ray {(3, 0, 2, 0) +AC1, 1, 0,0): A> 0}. 


Example 4.9 
Minimize -x,; —- xX) 
subjectto x - x 2 1 
—X) + XxX 2 1 
Xs xX, 2 0. 


This problem has no feasible solutions, as shown in Figure 4.4. Introduce the 
slack variables x3; and x4 and the artificial variables x, and xg. 
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x2 


x 


Figure 4.4. Empty feasible region. 


Multiply rows | and 2 by Mand add to row 0. 


2big-M 


Notice that the coefficients of M in the Zzpjy_;¢ Tow are all nonpositive. Hence, 


Phase I is complete; however, the artificial variables x; and x¢ are still positive. 


Therefore, we can stop here with the conclusion that the original problem is 
infeasible, even though this tableau for P(M) is not yet optimal. In fact, the 
reader can verify that by continuing the optimization process, we will detect that 
Problem P(M) is unbounded. Notwithstanding this, the infeasibility of the 
original problem is evident right at the present stage, and we can therefore 
terminate the optimization process. 


4.4 HOW BIG SHOULD BIG-M BE? 


There is another fundamental issue involved with the big—M method, 
namely, how big should M be? Clearly, given that the original problem is 
feasible, M should be large enough so that some basic feasible solution to P(M) 
with all artificial variables equal to zero has an objective value strictly better 
than the best basic feasible solution to P(M) that does not have all the artificial 
variables equal to zero (why?). Note that a finite value for such an M exists 
(why?). Hence, in selecting a value for M, it is erroneous to simply look at the 
magnitude of the objective function coefficients. One also needs to see how 
small (but positive) the sum of the artificial variables can get at a basic feasible 
solution to P(M). For example, consider the following problem P and the 
corresponding artificial problem P(M), where x3 is the slack variable, x4 is an 


artificial variable, and ¢> 0 is a (small) constant: 


Starting Solution and Convergence 173 


P: Minimize z= x, 
subjectto €x, - xX 2 € 
Xx, x, 2 0. 


P(M) : Minimize Zbig-M =X + Mx, 
subject to EX}, -— XX -— % + XY 
xy 2 x2 > X3 > X4 


E 
0. 


Vv Il 


Figure 4.5 depicts the feasible region in the (x1, x7} space. Problem P(M) has 
two basic feasible solutions, X and X, where x =(1, 0, 0,0) and x = (0, 0, 0, 
é). The first of these is (basic) feasible to P, but the second is infeasible to P. In 
P(M), the objective value for x is 1 and that for x is €M. We would therefore 
like EM > 1, that is M> 1/e. Observe that if we had picked M as, say, some 
large constant times the largest objective coefficient in P, this would have been 
inadequate by making ¢> 0 small enough. In Figure 4.5, we are comparing (0, 
0) with (1, 0), and although the former solution is infeasible, its infeasibility 
reduces with a reduction in ¢, thus requiring a higher penalty. In Exercise 4.43, 
we ask the reader to generalize this concept. 


4.5 THE SINGLE ARTIFICIAL VARIABLE TECHNIQUE 


Thus far, we have described two methods to initiate the simplex algorithm by 

the use of artificial variables. In this section, we discuss a procedure that 

requires only a single artificial variable to get started. Consider the following 
problem: 

Minimize ex 

subject to Ax 

x 


b 
0. 


Suppose that we can partition the constraint matrix A into A = [B, NJ], where B 
is a basis matrix, not necessarily feasible. B possibly corresponds to a set of 
variables forced into the basis, regardless of feasibility, in anticipation that these 
variables are likely to be positive at optimality. (Such a basis is called a crash 
basis.) Index the basic variables from 1 to m. 


IV Il 


XA 


v—E 
. 4 


Figure 4.5. Selecting a value for M. 
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Multiplying the constraints by B!, we get 
Ixp + B'Nxy =b 
where b = Bb. Suppose that b 2 0 (if b > 0, we have a starting basic 
feasible solution). To this system let us add a single artificial variable, x,, with a 
coefficient of -1 in each constraint that has 5; < 0. Call this column y,. This 
gives: 
Ixp + B'Nxy + y,x, =D. 


Now, introduce x, into the basis by selecting the pivot row r as follows: 


b, = minimum{é,} . 

i:b; <0 i 

Note that 6, < 0. On pivoting in row r (that is, inserting x, and removing x,), 
we get the new right—hand—side values: 


= -b(2 0) 
bh = b -b,(2 0), iff <Oi#r 
bi = 5 (2 0), otherwise. 


Thus by entering x, and exiting x, we have constructed a basic feasible solution 


to the enlarged system (the one including the single artificial variable). Starting 
with this solution, the simplex method can be used with either the two-phase or 
the big—M method. 


Note that the variable x, creates one extra dimension for this problem. 


Its column is the sum of the artificial variable columns we would have added for 
the two-phase or the big—M method. Hence, effectively, if we further restrict all 
the artificial variables in the two-phase or big—M method to be equal to each 
other, we would obtain the single artificial variable technique. Empirically, it 
has been found that the single artificial variable technique is not as efficient as 
the other two methods, perhaps because of its foregoing restrictive nature, 
whereby a simplex path needs to be traced in which all violated constraints are 
simultaneously satisfied. 


Example 4.10 
Minimize 2x, + 3x, 
subject to xy + xX 2 3 
—2x) + XxX 2 2 
Xx, PS) = 0. 


Subtracting the slack variables x3 and x, and multiplying by —1, the structural 
constraints become: 


-3 
= 2 


“xX — X2 + X3 
2x — X92 + X4 
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Appending a single artificial variable x; with activity vector (=) to the initial 


tableau of the Phase I problem, we get the following tableau: 


Xo xy x2 x3 X4 x5 RHS 


XX %. % %Xqy Xx RHS 


This tableau is ready for application of the two-phase method. Subsequent 
tableaux are not shown. 

An analysis of the two-phase method and the big—M method for the 
single artificial variable technique discussed in this section can be made similar 
to the analysis of Sections 4.2 and 4.3. The details are left to the reader in 
Exercise 4.19. 


4.6 DEGENERACY, CYCLING, AND STALLING 


Degeneracy causes several conceptual as well as computational difficulties in 
linear programming. Conceptually, we have seen in Chapter 3, for example, that 
in the presence of degeneracy, we have to be cautious about claiming the 
existence of alternative optimal solutions simply based on observing a zero 
reduced cost for a nonbasic variable at optimality. We also should be cautious 
about interpreting 0z/0b,; = w; as an actual (realizable) rate of change, and 


must reckon that not all bases representing an optimal solution necessarily 
satisfy z; —c; $0, j= 1, ..., 2 (for a minimization problem). 

Furthermore, we have seen in Chapter 3 that in the absence of degener- 
acy, the simplex method terminates finitely, either producing an optimal basic 
feasible solution or else verifying unboundedness. However, when a degenerate 
pivot occurs, we simply switch from one basis to another, both representing the 
same extreme point solution. This is evident in Table 3.1 by examining the 
situation before and after pivoting when b, = 0. As the process is repeated, it is 
conceivable that another degenerate pivot is performed, resulting in the same 
extreme point having a different basis representation. It is therefore possible that 
we may stay at a nonoptimal extreme point and pivot through a sequence of 
associated bases B,, B>,...,B,, where B, = B,. If the same sequence of pivots 


is used over and over again, we shall cycle forever among the bases B,, Bp...., 
B,= B, without reaching an optimal solution. 
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Hence, we present in the sequel some cycling prevention rules that 
guarantee finite convergence of the simplex algorithm by ensuring that no basis 
can be repeated. By doing so, such rules constructively demonstrate the 
following result that has so far been evident only in the absence of degeneracy. 


Theorem 4.1 


Given an optimal extreme point solution (for a minimization linear program), 
there exists an associated (optimal) basis for which z 7 7 ey <0 for all j = 1,..., . 


We begin our discussion by first presenting an example that demonstrates 
the phenomenon of cycling. 


Example 4.11 
(Cycling) 
Consider the following example given by E. M. L. Beale: 
Minimize — (3/4)xq + 20x, - (1/2)x6 + 6x7 
subject to x + (1/4)x4 - 8x5 - Xe + 9x7 
Xp + (1/2)xq - 12x53 - (1/2)xg + 3x7 
x4 + X6 
X, XQ, X34, X45 Xs, X6> x7 


i 


il 
Orono 


\V_ Il 


An optimal solution is given by x, = 3/4, x4 = x6 = 1, and all other variables 


equal to zero. The optimal objective value is —5/4. The following rules are 
adopted: The entering variable is that with the most positive z; — c;, and the 


leaving variable is determined by the minimum ratio test, where ties are broken 
arbitrarily. 


xy x2 x3 x4 x5 x6 x7 RHS 


oo O];—]}N 


-15/4 
0 
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Zz x x2 x3 X4 x5 X6 x7 RHS 


=s/2 
-1/4 16/3 
5/2 -56 


We see that the last tableau is identical to the first tableau. All the tableaux 
correspond to the extreme point (0, 0, 1, 0, 0, 0, 0), but with different bases 
representing this vertex. The foregoing sequence of pivots generated the bases 
B,. B, B;, B,, Bs, Bg, and B,, where By = B, a {a,, a, a3]. If the 
same sequence of pivots is used over and over again, the simplex algorithm will 
cycle forever among these bases without reaching an optimal solution. (This 
phenomenon of cycling can also occur at an optimal extreme point, without 
recognizing its optimality viaga tableau having z 7 ey £9 for all 7.) 


Observe that in the foregoing example, the variable x3; remained basic 


throughout the different tableaux, i.e., the last constraint in the problem 
remained inactive, and as such, we can delete the last constraint and derive an 
identical example that exhibits cycling. Now, as we shall see in Chapter 6, the 
resultant two-constraint problem (call it the primal problem) has an 
accompanying equivalent two-variable linear program (referred to as its dual 
problem) for which the bases are in a one-to-one correspondence. Hence, we can 
pictorially visualize the geometry of the cycling phenomenon by plotting the 
constraints of this dual problem in two-dimensions, and tracking the sequence of 
bases (intersection of pairs of lines in this case) that correspond to the cyclic 
loop of bases for the primal problem (see Exercise 6.75). The insight derived 
from this geometric view of cycling can also enable the construction of different 
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problem instances that exhibit the phenomenon of cycling (see the Notes and 
References section). 


There are several other ways by which we can construct alternative 
examples that exhibit cycling. One class of such examples are characterized by a 
permutation structure whereby, after every fixed number of iterations, the 
updated simplex tableau is a column permutation of the initial tableau. Exercise 
4.48 reveals the essence of this idea. Another approach is to directly examine 
the algebraic relationships that transition one tableau to the next and thereby 
enforce conditions under which an initial tableau would repeat after a sequence 
of iterations. The Notes and References section provides directions for further 
reading on constructing such cycling examples as well as on the geometry of 
cycling. 


Two Rules that Prevent Cycling 


Even though cycling appears to be very unlikely, it has been known to occur on 
practical real-world problems. Hence, it is of interest to develop rules that 
prevent cycling. We give two such rules here, and illustrate them in Example 
4.11. A validation of these rules is postponed until the next section. Consider the 
following linear programming problem: 


Minimize ex 
subject to Ax 
x 


b 
0, 


where A is an m x n matrix of rank m. Since the simplex method is usually 
started with the initial basis as the identity matrix (corresponding to slack and/or 
artificial variables), we shall assume that the first m columns of A form the 
identity. The following rule, which uniquely specifies the variable leaving the 
basis if the simplex minimum ratio test produces several candidates, will guar- 
antee noncycling. 


IV Il 


Lexicographic Rule for Selecting an Exiting Variable 
Given a basic feasible solution with basis B, suppose that the nonbasic variable 


xz 1s chosen to enter the basis (say, 0 < zx, — cy = maximum z; — c;). The 


index r of the variable xg_ leaving the basis is determined as follows. Let 


b, b; 
Ip = r:— > = minimum}— : yz > 0 s 
Vrk lsism | Vik 


If Ig is a singleton, namely Jp = {r}, then xg leaves the basis. Otherwise, 


Fico eile Sas, {24} 
1= : = minimum . 
Yrk ielg Vik 


form J, as follows: 
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If 7, is singleton, namely, 7, = {r}, then xp. leaves the basis. Otherwise, form 


Iy, where, in general, 7 F is formed from J j-1 as follows: 


Yrj as Vij 
[= r: = minimum)—— /;- 
\rk ielj_1 |Jik 


Eventually, for some j < m, 7; will be a singleton (why?). If 7; = {r}, then xB. 


leaves the basis. 

Before we illustrate the preceding rule, let us briefly discuss its 
implications. (The choice of the name for this method will become evident when 
we validate this rule in the next section.) In this rule, we first use the usual 
minimum ratio test as an exiting criterion. If this test gives a unique index, then 
the corresponding variable leaves the basis. In case of a tie, we try to break it by 
replacing the right-hand-side in the minimum ratio calculation by the first 
column y, and by only using the rows corresponding to the tie. If the tie is still 
not broken, the second column is used in the minimum ratio test, and so forth. 
When or before column m is reached, the tie must be broken, for if this were not 


the case, we would have two rows of the matrix B"! = (Y}, Y2.-->¥m) that are 
proportional (why?). This is impossible, however, in view of linear 


independence of the rows of BT. 


Example 4.12 


We now solve the problem of Example 4.11, using the additional rule for exiting 
from the basis. 


Z XM Xp Xz XG Xs X6: X7  RHS 
:C]0_0 0 34 0 2 +6] 01 
xy 0 
X 0 
x3 1 


Here, Jy = {1,2}, 7; = {2}, and therefore, XB. = Xz leaves the basis. Note that 


in Example 4.11, x; left the basis during the first iteration. 


Here, Jy = {3}. Therefore, xp, = x3 leaves. 
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The foregoing tableau gives the optimal solution, since z 77 ey £0 for all 


nonbasic variables. 


Bland’s Rule for Selecting Entering and Leaving Variables 


Another rule that prevents cycling has been suggested by Robert Bland. This is a 
very simple rule, but one that restricts the choice of both the entering and leaving 
variables. In this rule, the variables are first ordered in some sequence, say, x, 


X9,..X,, Without loss of generality. Then, of all nonbasic variables having 


zj;—Cj> 0, the one that has the smallest index is selected to enter the basis. 


Similarly, of all the candidates to leave the basis (i-e., which tie in the usual 
minimum ratio test), the one that has the smallest index is chosen as the exiting 
variable. 

Using this rule in Example 4.11, we see that the first four tableaux are 
produced as given. For example, in the first tableau, out of x4 and x6, we select 


X4 to enter the basis, and out of the candidates x, and x, for the exiting variable, 
we choose x, to leave the basis. However, in the fourth tableau, x, and x7 are 
eligible to enter the basis, and the present rule selects x, as the entering variable. 
The leaving variable is uniquely determined as x5. In Exercise 4.39 we ask the 


reader to verify that the resulting tableau leads to a nondegenerate pivot and that 
the pivots continue until optimality is attained. 


Some Practical Implementation Remarks and Stalling 


Although most real-world problems are known to be degenerate and the 
foregoing rules guarantee that cycling will not occur, these rules are largely 
ignored in most commercial codes for solving linear programming problems via 
the simplex method. There are two main reasons for this stance. First, the rules 
are either computationally expensive to implement, as in the case of the lexico- 
graphic method, or are computationally inefficient with respect to the length of 
the simplex path generated, as in the case of Bland’s Rule. Second, because of 
computer round—off errors, it is usually argued that the updated right—hand— 
sides are, in any case, perturbed from their actual values and one rarely 
encounters exact zero right-hand-side values. In fact, as we ask the reader to 
explore in Exercise 4.47, the lexicographic rule has an equivalent interpretation 
as a perturbation technique in which the original right-hand-side values are 
perturbed slightly to make the polyhedron nondegenerate. Accordingly, software 
packages typically adopt a “practical” anticycling rule based on appropriate 
perturbations of right-hand-sides of variable bounding constraints. However, 
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this is not guaranteed to obviate cycling (see the Notes and References section). 
It is important to note here that in the case of network structured linear 
programs, as we shall see later, neither of the foregoing reasons apply; however, 
cycling prevention rules are both easy to implement and are often 
computationally advantageous in this context. 

There is another issue that needs to be mentioned at this point. Although 
the foregoing rules prevent cycling, it is entirely possible that the simplex 
algorithm may go through an exceedingly long (though finite) sequence of 
degenerate pivots. In particular, for some classes of problems having a certain 
structure, it may be possible that the algorithm performs a number of consecu- 
tive degenerate pivots that are exponential in the size (m and n) of the problem. 
This phenomenon is known as stalling. The term arises because with increasing 
problem size, the algorithm can spend an enormous (though finite) amount of 
time at a degenerate vertex before finally verifying optimality or moving off 
from that vertex. Besides preventing cycling, we would also like to obviate 
stalling by ensuring that the length of a sequence of degenerate pivots is 
bounded from above by some polynomial in m and n. 

The key to preventing stalling appears to lie in what are known as stages. 
A stage is a sequence of degenerate pivots in which no nonbasic variable 
remains enterable throughout, with no strict subset of this sequence having the 
same property. We would like both the number of pivots in a stage, which is the 
length of a stage, and the number of stages in a sequence of degenerate pivots to 
be “small,” that is, polynomially bounded. The first of these two conditions is 
easy to achieve. For example, consider the following rule for choosing an 
entering variable, which is sometimes referred to as the Least Recently 
Considered (LRC) variable choice rule. Suppose that we maintain a circular list 
of the variables x,, x,....x,, and at any pivot, if a variable x, enters, then 


suppose that the next entering variable is selected as the first eligible candidate 
(with z; —c; > 0) from the list {x,4),--.X%,5 Xjs--.%4} Where x,4) = x1. Then it 


is clear that the length of a stage is no more than , since no variable could have 
remained enterable and have been ignored throughout a sequence of 1 (or more) 
degenerate pivots. Such a rule, by its nature, is aptly called an affirmative action 
policy. Note that Bland’s Rule is not an affirmative action strategy, since it 
“discriminates” in favor of variables having lower indices. In fact, there exist 
examples that demonstrate that Bland’s Rule admits stalling (see the Notes and 
References section of this chapter). On the other hand, it is an open question 
whether the lexicographic rule plus Dantzig’s Rule of entering the variable 


having the most positive z j ~ ¢; admits or prevents stalling for general linear 


programs. (For network structured problems, this is known to prevent stalling.) 

The issue of ensuring a polynomial bound on the number of stages in a 
sequence of degenerate pivots is harder to achieve. For general linear program- 
ming problems, a rule that provides such a guarantee has not as yet been 
discovered. However, as we shall see in Chapter 9, for network structured 
problems, a lexicographic cycling prevention rule does exist that is easy to 
implement, is computationally advantageous, and that prevents stalling when 
used in conjunction with an appropriate affirmative action entering rule. 
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4.7 VALIDATION OF CYCLING PREVENTION RULES 


In this section we show that the rules adopted in the previous section indeed 
prevent cycling. We do this by showing that none of the previous bases visited 
by the simplex method are repeated. In view of the finite number of bases, this 
automatically guarantees termination in a finite number of iterations. 
Lexicographic Rule 

In order to facilitate the proof of finite convergence under the lexicographic rule, 
consider the following notion of a lexicographically positive vector. A vector x is 
called /exicographically positive (denoted by x > 0) if the following two require— 
ments hold: 


1. xis not identically zero. 

2. The first nonzero component of x is positive. 
For example, (0, 2, -1, 3), (2, 1, -3, 1), and (0, 0, 1, -1) are lexicographically 
positive vectors, whereas (—1, 1, 2, 3), (0, 0, 0, 0), and (0, 0, —2, 1) are not. A 
lexicographically nonnegative vector, denoted by > 0, is either the zero vector 
or else a lexicographically positive vector. In order to prove that none of the 
bases generated by the simplex method is repeated, we first show that each row 


of the m x (m+ 1) matrix (b, B!) is lexicographically positive at each iteration, 
where b = B7'b. Basic feasible solutions that satisfy this property are sometimes 
called strongly feasible solutions. Indeed, in the absence of degeneracy, we have 
b > 0, and therefore, each row (b, B"!) is clearly lexicographically positive. 
First, recall that the original basis is I, and since b > 0, then each row of 
the matrix (b, B") = (b, I) is lexicographically positive. (If a feasible basis that 
is different from the identity is available, we still have a starting solution that 


satisfies the lexicographic positive condition -~ see Exercise 4.41.) In view of 
this, the preceding result will be proved if we can show the following: If each 


row of (b, B!) is > 0, then each row of (b, B!) is > 0 where B is the new 


basis obtained after pivoting and b = B 'b. Consider the following two tableaux 
before and after entering x,, and recall that the first m columns (ignoring the z 


column) in these tableaux represent B"! and BI, respectively (since the first m 


columns of the original problem form the identity). Here, Z denotes the 
objective value ¢gb before pivoting. 
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Before Pivoting 


Consider a typical row i of (b, B). From the foregoing tableau this row is 


given by 
~ 4, Yr y 
b, — VV — Vises Vin — yy |, fori zr (4.1) 
i Va ik > Fil Vek ik im Vie ik 
b 
et ope dd ge peper (4.2) 
Yrk rk rk 


Since y,, > 0 and the rth row is > 0 before pivoting, then from Equation (4.2), 


the rth row after pivoting is also > 0. Now consider i # r. There are two mutu- 
ally exclusive cases: either i ¢ Jp or else i € Jp. First suppose that i ¢ Ip. If 


Vik <0, then from Equation (4.1), we see that the ith row after pivoting is given 
by 
— Jj, — 
(Bj, Vite Vim) ~ LG Vets Vem) 
\rk 


which is the sum of two vectors that are lexicographically positive and 
lexicographically nonnegative (why?), and hence, it is > 0. Now, suppose that 


Vig > 0. By the definition of Jy and since i ¢ Ip, then b,/y,, < bj/y_, and 
hence, b; — (b,/¥,,)¥iq > 0. From Equation (4.1), the ith row is therefore > 0. 
Next, consider the case i € Jy. Then y, > 0 and b; ~ (b,/¥,4)¥iz = 0. There are 
two mutually exhaustive cases: either i ¢ J), or else 7 € J,. In the former case, 
by the definition of J), yj — (41/¥,-4)¥iR > O and thus from Equation (4.1), the 
ith row is > 0. If, on the other hand, i € /,, then yj — (9)/¥,4) vig = 0 and we 


examine whether i € J, or not. This process is continued at most m + | steps, 
with the conclusion that each row of (b, B) is > 0. 


The foregoing analysis shows that each row of (B"'b, B!) is lexico- 
graphically positive at any given iteration. This fact will be used shortly to prove 
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finite convergence. First, by examining row 0 before and after pivoting, note 
that 


-1 -1 5-1 poly — 2k ~ ok (Z 
(cgB b,cgB ) — (cgB b,c ,B ) = Sy Cr ea Mraes om): 
nr 


Note that (B,, yp15---:¥ym) is the rth row of (b, B~') and is therefore > 0. Since 
Zp — cy > Oand y,, > 0, it is therefore evident that 


(cgB"'b,cgB"') - (¢,B"'b,c,B"') > 0. 


We are now ready to show that the lexicographic rule of Section 4.6 will 
indeed prevent cycling. We do this by showing that the bases developed by the 
simplex method are distinct. Suppose by contradiction that a sequence of bases 
B,, B3,...,B, is generated where B, = B,. From the preceding analysis we have 


-l -l1 -l -l a 
(cx,Bj b,¢z Bj -(ca,,,Bybsen,,,B711] ~ 0, for j = eee t-1. 


Adding over j = 1,...,£— 1 and noting that B, is assumed equal to B,, we get 0 
> 0, which is impossible. This contradiction asserts that the bases visited by the 
simplex algorithm are distinct. Because there are but a finite number of bases, 
convergence in a finite number of steps is established. 


Bland’s Rule 


In the absence of degeneracy, the objective function itself is a strictly decreasing 
monotone function, which guarantees that no basis will be repeated by the 
simplex algorithm. On the other hand, the lexicographic rule ensures finite con- 


vergence by showing that while c 3B 'b may remain constant in the presence of 


degeneracy, the vector (cpB 'b, zB’) is lexicographically monotone de- 
creasing. Bland’s rule has the following monotone feature: In a sequence of 


degenerate pivots, if some variable x, enters the basis, then x, cannot leave the 


basis until some other variable having a higher index than g, which was non— 


basic when Xq entered, also enters the basis. If this holds, then cycling cannot 


occur because in a cycle, any variable that enters must also leave the basis, 
which means that there exists some highest indexed variable that enters and 
leaves the basis. This contradicts the foregoing monotone feature (why?). 

To show that this monotone feature holds true, consider the basic feasible 
solution at which x, enters. Let J; and J3 be the sets of the current nonbasic 
variables having indices respectively less than and greater than g. As in 


Equation (3.5), we can write the representation of the linear program in the 
current canonical form as follows: 
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Minimize > € jx; +E,x, + > C;x; 


sed, - vib) a 
subject to Axy +Xp= b (4.3) 
Xv, Xp 29, 
where Xg and xj are the current basic and nonbasic variables and c; = c; — Z; 
for all j. Note that since x, is an entering variable by Bland’s rule, we have that 


Cy <Oand c; >0 for j € J. 


Now, suppose on the contrary that we go through a sequence of degen- 
erate pivots in which x pp J € Jo, remain nonbasic, and we reach a pivot in 


which some x, enters and x, leaves the basis. Let us treat Equation (4.3) as an 


P 
“original” linear program, that is, define variable columns a,, bases, and cost 


coefficients with respect to this representation. In particular, let the basis in 
Equation (4.3) at the pivot in which X, enters and x, leaves be denoted by B, 


with the basic cost coefficient vector being ¢g. Hence, tp, By! an — Cp > 0, 


since x, enters. Because ¢, > 0 (why?), this means that €g By! a, > 0. Let 


Pp 


ee Be a, and denote the pivot element in the row for the currently basic 


variable x, by Yap: Note that y,, > 0 and c, <0, so that in the positive inner 
product cay p> the component y,,c, gives a negative contribution. Hence, 
there must be another basic variable x, that has c, > 0 and y,, > 0, since other 
than Cy: 
Xg-variables, all of which have nonnegative c— values. In particular, we must 


all other components in ¢g, correspond either to x;, j € J}, or to the 


have r € J; (why?), which also means that x, is currently zero, since we have 
been performing degenerate pivots. But this means that x, is also an eligible 


candidate to leave the basis in the minimum ratio test. Since r < q, the variable 


Xq cannot be the leaving variable by Bland’s Rule. This contradiction proves the 


desired result. 
Other Anticycling Rules 


There are several alternative rules for selecting entering and exiting variables that 
can be devised in order to preclude the phenomenon of cycling in the simplex 
method. For example, S. Zhang showed that the following type of “last in, first 
out; last out, first in” rule would prevent cycling. Of all candidates to enter the 
basis, select the one that became nonbasic most recently. Likewise, of all 
candidates for the exiting variable, select the one that became basic most recently. 
(In either case, ties to the rule may be broken arbitrarily.) Unfortunately, this rule 
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also admits stalling, although it inspires a wide class of anticycling rules (see the 


Notes and References section). 


EXERCISES 
[4.1] Consider the following linear programming problem: 
Maximize —-x; + 3x 
subjectto x + X9 
2x, + 3x, 
x2 
> x2 


Solve the problem graphically. 


IV IA IA IV 


CNA 


Solve the problem by the two-phase simplex method. Show that the 


points generated by Phase I correspond to basic solutions of the 


original system. 


[4.2] Solve the following problem by the two-phase simplex method: 


Minimize 


x + 3xp -— x3 
subjectto 2x, + Xx + 3x, 2 3 
xX, + A) > | 
—Xy - 5X5 + x3 < 4 
Xx, Xo, x3 2 0. 


[4.3] Solve the following problem by the two-phase simplex method: 


Maximize 2x} - x, + % 

subject to 3x, + Xx, -— 2x3, < 8 
4x, — Xp + 2x3 > 2 
2x; + 3x» = 2x3 => 4 
xX, Xp, x 2 0. 


[4.4] Solve the following problem by the two-phase method: 


Maximize 4x, + 5x, - 3x; 
subjectto x, + 2x, + % 
a eae” 
xX + 3x9 + & 
Xs 723 %3 


10 
6 
14 
0. 


IV IA IV Tl 


[4.5] Solve the following problem by the two-phase method: 


Maximize —x, — 2x 
subject to 3x, + 4x, 
ie Cr? 
> *2 


IV VIA 


12 
2 
0. 


[4.6] Solve the following problem by the two—phase method: 


Maximize 5x, 


a 2x9 eet x3 
subjectto 2x, + 4% + %& 
2x) + 2x. + 3x3 

™> x2 


x3 


unrestricted. 
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[4.7] Phase I of the two—phase method can be made use of to check redundancy. 
Suppose that we have the following three constraints: 


xy 2x5 2D 
xX + 3X4 => 4 
2x + x2 > 6. 


Note that the third constraint can be obtained by adding the first two constraints. 
Would Phase I detect this kind of redundancy? If not, what kind of redundancy 
will it detect? Does the type of redundancy in which some inequality holds 
whenever the other inequalities hold imply degeneracy? Discuss. 


[4.8] Show how Phase I of the simplex method can be used to solve 1 
simultaneous linear equations in m unknowns. Show how the following cases 
can be detected: 


a. Inconsistency of the system. 
b. Redundancy of the equations. 
c. Unique solution. 


Also show how the inverse matrix corresponding to the system of equations can 
be found in Part (c). Illustrate using the following system: 


xX + 2x, - x3 = 4 
—X1 — Xp + 3x3 = 3 
3x a 5X2 = 5x3 = 5, 


[4.9] Solve the following problem by the big—M method: 
Minimize 3x, + 2x, + 4x; + 8x4 


subjectto x, — 2x, + 3x3; + 6x4 2 
—2x, + 5x, + 3x3 - 5x4 < 3 
xX, X25 X3; X4 > 0. 


[4.10] Use the big—M method to solve the following problem: 


Minimize -2x, + 2x, + x3 + X4 


subjectto x, + 2x, + 2x3 + X4 < 2 
Xo 2x2 + x3 a 2X4 > 3 
2x, as x4 am 3x3 => 2 
X15 x9, X3; X4 > 0. 
[4.11] Solve the following problem by the big—-M method: 
Maximize 2x, - x) 
subject to 2x, + 3x, < 6 
xX, + x2 > 1 
xX; x, 2 0. 
[4.12] Solve the following problem by the big—M method: 
Maximize 2x, + 4x, + 4x, - 3x4 
subject to 2x, + XxX, + x3 = 4 
xX + 4x + 3x, = 6 
xX, x2, X35 X4 > 0. 
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[4.13] Solve the following problem by the big—M method: 


Minimize x + 4x, -— XxX 
subjectto —2x, + 2x, - x3 + 2x4 < 2 
2x; + 3x + 2X3 —- 2x 4 = 4 
x} —- 3x3 + Xx 2 2 
x, X2, xy 2 0. 


X3 unrestricted. 


[4.14] Use the big—/ method to solve the following problem: 


Maximize x} — 2x2 + %3 

subjectto x, + 2x, -— % 2 4 
xX - 4x. + x3 5 2 
xX, xX, x 2 0. 


[4.15] Solve the following problem by the big—M method: 


Maximize 5x, - 2x, + 4% 

subjectto x + 4% + »% < 5 
2x, + X% + 3%, 2 2 
xX; > 0. 


x3 
X unrestricted, 


[4.16] Solve the following linear program by both the two-phase method and 
the big—M method: 


Minimize 3x, -— 3x2 + x3 
subjectto x, + 3x, - 2x3 2 5 
-3x, -— 2x. + x3 <5 4 
xX, Xo; x3 2 0. 


[4.17] Use the single artificial variable technique to solve the following linear 
programming problem: 


Minimize —x, — 2x, + x3 
subjectto x + 2x, + x3 2 4 
2x, -— % 2 3 
Xn + X3 < 2 
xX; x2; x3 > 0. 


[4.18] Use the single artificial variable technique to solve the following problem: 


Maximize 4x, + 5x7 + 7x3 - X%%4 
subjectto x + X, + 2x3 - xy 2 1 
2x; = 6x, + 3x3 + X4 < -3 
2x, + 4x. + 2x3 + 2x4 = -5 
x1, x2; X4 2 0 


x3 unrestricted. 


[4.19] Discuss in detail all the possible cases that may arise when using the 
single artificial variable technique with both the two-phase method and the big— 
M method. 


[4.20] Discuss the advantages and disadvantages of using a single artificial 
variable compared with a method using several artificial variables. 
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[4.21] Is it possible that the optimal solution value of the big—M problem is 
unbounded and at the same time the optimal solution of the original problem is 
bounded? Discuss in detail. 


[4.22] Is it possible that the region in R” given by 


Ax = b 
x 2 0 
is bounded, whereas the region in R”*” 
Ax+x, = Db 
x, X, 2 0 


is unbounded? What are the implications of your answer on using the big—M 
method as a solution procedure? 

[4.23] Suppose that either Phase I is completed or the bounded optimal solution 
of the big—M problem is found. Furthermore, suppose that there exists at least 
one artificial at a positive level indicating that the original system Ax = b and 
and x > 0 has no solution. How would you differentiate between the following 
two cases? 


a. The system Ax = b is inconsistent. 
b. The system Ax = b is consistent but Ax = b implies thatx 2 0. 


Illustrate each case with an example. 
[4.24] Suppose that the big—M/ method is used to solve a minimization linear 
programming problem. Furthermore, suppose that z, —c, = maximum 
(z; — ¢;) > 0. Show that the original problem is infeasible if not all artificials 


are equal to zero and yj, <0 for each i such that xp, is an artificial variable. 


[4.25] Geometric redundancy occurs when deletion of a constraint does not alter 
the feasible set. How can geometric redundancy be detected? (Hint: Consider the 
objective of minimizing x,, where x, is a particular slack variable.) 

Geometrically 


redundant 
constraint 


v 


[RSS Ses eeeR 


[4.26] Suppose that at some iteration of the simplex method the slack variable 
xX, is basic in the ith row. Show that if ae 0,7 = 1, .., 1,7 #.s, then the 


constraint associated with x, is geometrically redundant. 
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[4.27] Suppose that it is possible to get the constraints of a linear program to the 
form Ixp + B-'Nxy = b, where b = B™'b 2 0. Show that a single artificial 


variable x, with activity vector b (where b< b) can be added and a basic 
feasible solution can be obtained thereby. 


[4.28] A manufacturer wishes to find the optimal weekly production of items A, 
B, and C that maximizes the profit. The unit profit and the minimal weekly 
production of these items are, respectively, $2.00, $3.00, and $4.00, and 100 
units, 80 units, and 60 units. Items A, B, and C are processed on three machines. 
The hours required per item per machine are summarized below. 


ITEM 
Machine A B C 
1 0 1 2 
2 1 1 1 
3 2 1 1 


The numbers of hours of machines 1, 2, and 3 available per week are 240, 400, 
and 380, respectively. Find an optimal production schedule. 


[4.29] A manufacturer of metal sheets received an order for producing 2500 
sheets of size 2' x 4' and 1000 sheets of size 4' x 7'. Two standard sheets are 
available of sizes 11' x 3000' and 10' x 2000'. The engineering staff decided that 
the four cutting patterns in the figure are suitable for this order. Formulate the 
problem of meeting the order and minimizing the waste as a linear program and 
solve it by the simplex method. 


Patterns Patterns 


from size 11' x 3000' from size 10' x 2000' 
Pattern 1 Pattern 2 
I 4 7 4 £ 4 
2 
4 
2 
Pattern 3 Pattern 4 
ome 2 — 7 4 4 * 4 4 


eaeaen Wee 


[4.30] A trucking company owns three types of trucks: type I, type II, and type 
III. These trucks are equipped to haul three different types of machines per load 
according to the following chart: 


TRUCK TYPE 
I II I 
Machine A 1 1 1 
Machine B 0 1 We 
Machine C 2 1 1 
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Trucks of type I, Il, and III cost $500, $600, and $1000 per trip, respectively. 
We are interested in finding how many trucks of each type should be sent to 
haul 12 machines of type A, 10 machines of type B, and 16 machines of type C. 
Formulate the problem and then solve it by the simplex method. (This is an 
integer programming problem; you may ignore the integrality requirements.) 


[4.31] A company produces refrigerators, stoves, and dishwashers. During the 
coming year, sales are expected to be as shown below. The company wants a 
production schedule that meets the demand requirements. Management also has 
decided that the inventory level for each product must be at least 150 units at the 
end of each quarter. There is no inventory of any product at the start of the first 
quarter. 


QUARTER 
PRODUCT 1 2 3 4 
Refrigerators 1500 1000 2000 1200 
Stoves 1500 1500 1200 1500 
Dishwashers 1000 2000 1500 2500 


During a quarter only 19,000 hours of production time are available. A 
refrigerator requires 3 hours, a stove 5 hours, and a dishwasher 4 hours of 
production time. Refrigerators cannot be manufactured in the fourth quarter 
because the company plans to modify tooling for a new product line. 

Assume that each item left in inventory at the end of a quarter incurs a 
holding cost of $5. The company wants to plan its production schedule over the 
year in such a way that meets the quarterly demands and minimizes the total 
inventory cost. Formulate the problem and then solve it by the simplex method. 


[4.32] A manufacturer wishes to plan the production of two items A and B for 
the months of March, April, May, and June. The demands that must be met are 
given below: 


MARCH APRIL MAY JUNE 
Item A 400 500 600 400 
Item B 600 600 700 600 


Suppose that the inventory of A and B at the end of February is 100 and 150, 
respectively. Further suppose that at least 150 units of item B must be available 
at the end of June. The inventory holding costs of items A and B during any 
month are given by $1.20 and $1.00 times the inventory of the item at the end of 
the month. Furthermore, because of space limitation, the sum of items A and B 
in stock cannot exceed 250 during any month. Finally, the maximum number of 
items A and B that can be produced during any given month is 550 and 650, 
respectively. 


a. Formulate the production problem as a linear program. The objective 
is to minimize the total inventory cost (the production cost is 
assumed constant). 

b. Find an optimal production/inventory pattern. 

c. Management is considering installing a new manufacturing system 
for item B at the end of April. This would raise the maximum items 
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that can be produced per month from 650 to 700, and meanwhile 
would reduce the unit manufacturing cost from $8.00 to $6.50. 
Assess the benefits of this system in reducing the total manufacturing 
costs plus inventory costs. If you were a member of the management 
team, discuss how you would assess whether the new system is cost— 
effective. 

d. Suppose that management decided to introduce the new system. Market 
research indicated that item B can be backlogged without serious 
dissatisfaction of customers. It was the management’s assessment that 
each unit of unsatisfied demand during any month must be charged an 
additional $1.00. Formulate the production/inventory problem and find 
an optimal solution by the simplex method. 


[4.33] A company manufactures stoves and ovens. The company has three 
warehouses and two retail stores. Sixty, 80, and 50 stoves and 80, 50, and 50 
ovens are available at the three warehouses, respectively. One hundred and 90 
stoves, and 60 and 120 ovens are required at the retail stores, respectively. The 
unit shopping costs, which apply to both the stoves and ovens, from the 
warehouses to the retail stores are given below: 


STORE 
WAREHOUSE 1 2 
1 4 5 
2, 2 4 
3 5 3 


Find a shipping pattern that minimizes the total transportation cost by the 
simplex method. 


[4.34] A farmer has 200 acres and 18,000 man-hours available. He wishes to 
determine the acreage allocated to the following products: corn, wheat, okra, 
tomatoes, and green beans. The farmer must produce at least 250 tons of corn to 
feed his hogs and cattle, and he must produce at least 80 tons of wheat, which he 
has precontracted. The tonnage and labor in man-hours per acre of the different 
products are summarized below: 


CORN WHEAT OKRA — TOMATOES BEANS 
Tons/acre 10 4 4 8 6 
Man-hours/acre 120 150 100 80 120 


The corn, wheat, okra, tomatoes, and beans can be sold for $120.00, $150.00, 
$80.00, $60.00, and $75.00 per ton. Find an optimal solution. 

[4.35] Solve the following problem, using the lexicographic rule for noncycling. 
Repeat using Bland’s Rule: 


Maximize x4 + 2x, + 3 

subjectto x + 4x. + 3x3; < 4 
—H + xX + 4x3 <1 
yot 3x2 + x3 < 6 
xX, x2, x3 > 0. 


{4.36] Consider the following region: 


194 Chapter 4 


2x, -— 2X, + % S$ 4 
“Xx +t XX -— % < 3 
4x, + X, — 3x3 < 2 
xX; x7; x3 > 0. 


Recall that d is a direction of the region if Ad < 0, d > 0, and d is nonzero. Thus, 
in order to find directions of the region, we may solve the following problem: 


Maximize d,; + d, + a, 


subjectto 2d; - 2d, + d3 < 0 
-d| + a, = dy < 0 
4d, + d, —- 3d, < 0 
da + a + 3 < 1 
d, a, a, >0 


The constraint d, + d, + d3 < 1 is added for normalization; otherwise, the 
optimal objective may reach +00. Solve this direction—finding problem by the 
simplex method with the additional lexicographic exiting rule. Does this 
procedure generate extreme directions? Why or why not? Can the normalization 
constraint ad; + dy + d3 < 1 be deleted? If so, describe how to find directions 
if the simplex method indicates unboundedness. Illustrate by deleting this con— 
straint and resolving the problem. 

[4.37] Consider the following problem: 


Maximize 2x, + 3x, 


subjectto x + 2x, < 10 
xX + 2X4 < 6 
x} + Po) < 6 
x, X, 2 0. 


a. Solve the problem graphically and verify that the optimal point is a 
degenerate basic feasible solution. 

Solve the problem by the simplex method. 

c. From Part (a), identify the constraint that causes degeneracy and 
resolve the problem after deleting this constraint. Note that 
degeneracy disappears and the same optimal solution is obtained. 

d. Is it true in general that degenerate basic feasible solutions can be 
made nondegenerate by deleting some constraints without affecting 
the feasible region? 


[4.38] Show that cycling can never occur, even in the presence of degeneracy, 
provided that a unique minimum is obtained in the computation 


b; 
minum) > Vig > o| 


l<ism i 


where b = Bb, yz, = B !a,, and x, is the entering variable. 


[4.39] Solve the problem in Example 4.11 using Bland’s cycling prevention rule. 
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[4.40] Suppose that we have an optimal extreme point of a minimization linear 
programming problem. In the presence of degeneracy, is it possible that this 


extreme point corresponds to a basic feasible solution such that z; — c; > 0 for at 


least one nonbasic variable? If this were the case, are we guaranteed of another 


basic feasible solution corresponding to the same extreme point where z; — c; < 
0 for all nonbasic variables? Why or why not? Illustrate by a numerical example. 


[4.41] In order to prove finite convergence of the simplex method using the 
lexicographic rule, we assumed that the first m columns of the constraint matrix 
form the identity. Show that this assumption can be relaxed provided that we 
have any basic feasible solution. (Hint: Let B be the starting basis and consider 
the following equivalent problem.) 

Minimize Oxp + (ey - cpB 'N)xy 


subject to Ixp + B Nx, =B'b 
XB> XV = 0. 


[4.42] We showed that the row vector (cpB 'b, c ah) is lexicographically 
decreasing from one iteration to another using the lexicographic rule. Give an 
economic interpretation of this fact. (Hint: Note that z = cpB!b and that 
éz/db = ¢,B!.) 

[4.43] Let X= {x: Ax =b, x > 0} be nonempty, where A is m x n. Consider the 
preemptive priority multiple objective problem of maximizing ¢)x over x € X, 
and among all alternative optimal solutions, maximizing ¢.x, and among all 
continuing alternative optimal solutions, maximizing ¢3x, and so on, until the 
final problem of maximizing ¢,x over continuing alternative optimal solutions. 
Call any such resulting optimal solution x” a preemptive optimal solution, and 
assume that one exists. Show that there exists a scalar My > 0 such that for any 
M> Mo, a solution x” is a preemptive optimal solution if and only if it solves 
the following problem: 

r * 
Maximize) $ M’ex:xe x. 


i=] 


[4.44] Suppose that a linear programming problem admits feasible points. 
Utilize the result of Exercise 4.43 to show that if Mis large enough, then a finite 
optimal solution of the big—M problem must have all artificials equal to zero. 
Give all details. 


[4.45] Consider the linear programming problem to minimize cx subject to 
x € X = {x: Ax =b, x > 0}, where X is a nonempty, bounded polyhedral set in 


R”. Define @, < 6) <-- < 4, as the distinct values taken on by ex over the 
extreme points of X and let S; be the set of extreme points of X for which the 
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objective value is 9; for i = 1,..., 7. An arrangement of the extreme points in S, 
followed by those in S,, then S3,...,5, is said to be a ranking of the extreme 
points of X with respect to the objective function ex. Suppose that x),...,X, is a 


partial ranking of the extreme points. Show that a candidate for the next ranked 
extreme point x,,, is adjacent to some vertex in x, ,...,x,;. How would you use 


this result to produce all the sets S),..., 5, ? 


[4.46] Consider the following problem: 

Minimize (ex,cY) 

subject to A(x, Y) = (b, 1D 

(x,¥) > 0 

where A is anm  n matrix, ¢ is an n-vector, and the variables are the n-vector x 
and the n x m matrix Y. The objective function is a row vector, and the 
minimization is taken in the lexicographic sense, that is, (ex, cY2) < (ex), 
cY,) if and only if (ex,, c¥;) — (ex2, cY,) > 0. Each row of the matrix (x, Y) is 
restricted to be lexicographically nonnegative, which means that each row is 
zero or > 0. 


a. Let x be a basic feasible solution of the system Ax = b, x > 0 with 


-l -1 
basis B. Show that x = ee >) and Y = i is a feasible solution 


of the foregoing problem provided that (Bob, B!) > 0. 


b. Show that the simplex method with the lexicographic exiting rule of 


Section 4.6 generates the sequence (x), Y,), (x2, Y2) ..., where 
(ex ;_1, CY;_1) — (ex;, c¥;) > 0 for all. Interpret this fact empha— 
sizing the relationship between the bases generated by the simplex 
method and the foregoing problem. 


[4.47] Consider the following problem: 
Minimize cx 
subjectto Ax = b 
x 2 0. 


Assume that the first m columns of A form the identity and assume that b > 0. 


Given a basis B, the corresponding feasible solution is nondegenerate if B! b> 
0. Consider the following perturbation procedure of Charnes. Replace b by b + 


va jel where ¢ is a very small positive number. Now, suppose that we have 


a basis B, where B™! (b + Dm aje/) =b+ Sry > 0. Suppose that x, is 


chosen to enter the basis and the following minimum ratio test is used: 


Starting Solution and Convergence 197 


b + &X yye? 
ee j=l . 
minimum 4 ——~———— : yg > >. 
\<i<m Vik 


a. Show that the minimum ratio occurs at a unique index r for a 
sufficiently small ¢ Show that the method of finding this index is 
precisely the rule of Section 4.6. 

b. Show that the new right-hand-side after pivoting is positive and that 
the objective function strictly improves even in the presence of 
degeneracy in the original problem. 

c. Show that cycling will not occur if the rule in Part (a) is adopted. 
Interpret this in terms of the perturbed problem. 

d. Show that all the computations can be carried out without explicitly 
replacing the right-hand-side with b+ 7a jé. and without 
explicitly assigning a value to «. 


[4.48] Consider an initial tableau for a maximization linear programming problem 
in the following form, where B is a 2 x 2 nonsingular matrix, ¢p is a 2-vector, and 


nis even: 


Xn+] Xn+2 RHS 


a1 


7 a) 
2 
—c,(1+ B+ B*) 
B 


Furthermore, suppose that 


ie n 
B2) =I and -c,(1+B+B* +---+B2)=(0, 0). 

(a) Display the form of the tableau for the cases when the basic variables are 
selected as (i) {x3, x4}; (ii) {x5, x6}, and finally, (iii) {x,41, X,42}. In 
each case, verify that the updated tableau is a (block-wise) permutation of 
the columns of the initial tableau. 


(b) Explain how the phenomenon observed in Part a can lead to an instance 
where the above linear program would cycle, starting and ending at the 
basic feasible solution displayed above. 


(c) Illustrate Part b using n = 8, eg = (1/2, -7/20). 
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and with the simplex cycle proceeding through the following closed loop of 
bases: {x,, Xp}, {X9, x3}, {x3, x4}, {X4, x5}, {x5, Xe}; tee {Xg, Xiq}5 
{x}, %jq}, and back to {x,, x2}. 


NOTES AND REFERENCES 

1. The use of artificial variables to obtain a starting basic feasible solution 
was first published by Dantzig [195 1a]. 

2: The single artificial variable technique of Section 4.5 can be viewed as 


the dual of a similar technique that adds a new row to obtain a starting 
basic dual feasible solution. The latter is discussed in Section 6.6. Com- 
putational results appear in Wolfe and Cutler [1963]. 


3. A general Phase I method in which the sum of infeasibilities is reduced 
without regard to the feasibility of individual variables has been sug- 
gested by Maros [1986]. 


4. The cycling example of Section 4.6 is due to Beale [1955]. The proof of 
the cycling prevention rule via lexicographic ordering was published by 
Dantzig, Orden, and Wolfe [1955]. The cycling prevention rule can also 
be interpreted as a perturbation technique, as briefly described in Exercise 
4.47. This technique was independently devised by Charnes [1952]. 


Ey Bland’s cycling prevention rule [1977] is simple and elegant, although 
computationally unattractive. The notion of stages and an example of a 
network structured linear program for which Bland’s rule admits stalling 
has been provided by Cunningham [1979]. The term “affirmative action” 
for entering variable selection strategies is from Fathi and Tovey [1986]. 
For stalling prevention in network flow problems using Dantzig’s 
entering variable choice and the lexicographic cycling prevention rule, 
see Orlin [1985]. Marshall and Suurballe [1969] show that the smallest 
examples that can exhibit cycling have m = 2, n = 6, (n — m) > 3, and 
cycle length equal to 6. Related results for network structured problems 
are studied by Cunningham [1979]. For a practical anticycling rule, see 
Gill et al. [1988]. Hall and McKinnon [2004] have shown, however, that 
this technique does not obviate cycling, using a class of simple examples. 
The LIFO (last—in—first—out) type of anticycling rule has been proposed 
by Zhang [1991], who also develops a wide class of cycling prevention 
rules that uphold certain consistent and well—preserved orderings of 
variable indices within the pivot rule. 


6. The cycling examples based on a permutation structure as explained in 
Section 4.6 are due to Zomnig [2008] (Exercise 4.48 provides some details 
of this concept based on this paper). Also, see Zornig [2006] for a 
discussion on constructing cycling examples using Dantzig’s entry rule 
and the steepest edge entry rule (see Chapter 5) based on direct algebraic 
pivoting relationships. Lee [1997] illustrates the geometry in the 
requirement space (see Chapter 2) for the dual (see Chapter 6) to 
Hoffman’s [1953] example of cycling. In fact, Hoffman’s [1953] example 
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also conforms to the abovementioned permutation structure. Furthermore, 
Avis et al. [2008] discuss the geometry of cycling from the viewpoint of 
the sequence of bases for the dual problem (see Chapter 6 and Exercise 
6.75) that correspond to the closed cyclic loop of bases in a given 
(primal) problem. This viewpoint also affords a mechanism for 
constructing examples that exhibit the phenomenon of cycling. 


7, Exercise 4.43 addresses the notion of equivalent weights for preemptive 
priority multiple objective programs, as discussed in Sherali and Soyster 
[1983a] and in Sherali [1982]. 


8. Exercise 4.45 states the basic result of Murty [1968] used for ranking 
extreme points of polytopes. An alternative ranking technique appears in 
Sherali and Dickey [1986]. For the special case of ranking vertices of an 
assignment polytope, see Bazaraa and Sherali [1981]. Also, for related 
work on enumerating all vertices of polytopes, see Mattheiss [1973] and 
Mattheiss and Rubin [1980]. 


This page intentionally left blank 


FIVE: SPECIAL SIMPLEX 
IMPLEMENTATIONS AND 
OPTIMALITY CONDITIONS 


In this chapter we describe some special implementations of the simplex procedure 
or slight modifications of it. The formats considered here will prove advantageous 
in later chapters. The revised simplex method, which proceeds through the same 
steps as the simplex method but keeps all pertinent information in a smaller array, 
is described in Section 5.1. Numerically stable forms of this method are also 
discussed. In Section 5.2 we describe a slight modification of the simplex method 
for dealing implicitly with lower and upper bounds on the variables without 
increasing the size of the basis. The remainder of the chapter is devoted to some 
geometric aspects of the simplex method. In particular, Farkas’ Lemma and the 
Karush—Kuhn-Tucker optimality conditions are discussed. 


5.1 THE REVISED SIMPLEX METHOD 


The revised simplex method is a systematic procedure for implementing the steps 
of the simplex method using a smaller array, thus saving storage space. Let us 
begin by reviewing the steps of the simplex method. 


Steps of the Simplex Method (Minimization Problem) 


Suppose that we are given a basic feasible solution with basis B (and basis inverse 
B! ). Then: 


1. The basic feasible solution is given by xg = B'b = b and Xy = 0. 
The objective z = c,B 'b = pb. 
2. Calculate the simplex multipliers w = cpB. For each nonbasic 


j — -1 _— = — . = — 
variable, calculate z; — c; = ¢gB a; — c; = wa; — c;. Let zy — cy 


maximum z; — c;. If zg —c, < 0, then stop; the current solution is 
optimal. Otherwise, go to Step 3. 

3. Calculate y,; = Bla. If y, <0, then stop; the optimal objective 
value is unbounded. Otherwise, determine the index of the variable xp. 


leaving the basis as follows: 


i iia > Vig > o. 


Vrk 1<i<m Nik 


Update B by replacing a B, with a, and return to Step 1. 
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Examining the preceding steps, it becomes clear that the simplex method 
can be executed using a smaller array. Suppose that we have a basic feasible 


solution with a known B™!. The following array is constructed where w = ¢ 3B 
and b = Bb: 
BASIS INVERSE RHS 


Note that the foregoing tableau, called the revised simplex tableau, is also present 
in the usual simplex tableau, provided that we start with the BASIS INVERSE 
section having a zero vector in the objective row and an identity matrix in the other 
rows. (This may correspond to the slack variables, if available as a starting basis, 
or may simply be a section of this type maintained for bookkeeping purposes.) The 
RHS is the usual tableau right-hand-side. Now, imagine that we are performing 
the simplex algorithm on the entire tableau, but we are keeping hidden all but this 
array and any other required information. Hence, only the “exposed” information is 
explicitly calculated. The first piece of information that we need to see is the set of 
(z pee j-values. Since w is known, these values can be explicitly calculated as in 


Step 2 to check for optimality. Suppose that z, — c, > 0. Then we wish to examine 
the updated column of x;. Using B_!, we may compute Yyi= B la,. If y, <0, we 


stop with the indication that the optimal objective value is unbounded. Otherwise, 
the updated column of x; can be appended to the revised simplex tableau as 


shown, while the rest of the tableau is still kept “hidden.” 


BASIS INVERSE RHS XE 


The index r of Step 3 can now be calculated by the usual minimum ratio test. More 
importantly, pivoting at y,, gives as usual the new values of w, B |b, ande ab, 
and the process is repeated. We leave it as an exercise to the reader to rigorously 
verify that pivoting indeed updates the (m+ 1) x (m+ 1) revised simplex tableau. 
The revised simplex method converges in a finite number of steps 
provided that a cycling—prevention rule is adopted as discussed in Chapter 4. This 
is obvious since the revised simplex method performs exactly the same steps as the 
simplex method, with the exception that only a part of the tableau is explicitly 
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maintained and other information is generated only as required. The following is a 
summary of the revised simplex method. 


Summary of the Revised Simplex Method in Tableau Format (Minimization 
Problem) 


INITIALIZATION STEP 


Find an initial basic feasible solution with basis inverse B™!. Calculate w = 
c 3B, b = Bb, and form the following array (revised simplex tableau): 


BASIS INVERSE RHS 


MAIN STEP 
For each nonbasic variable, calculate z; —c; = wa; — c;. Let z, —c = 
maximum z; — c;. If z, —c, < 0, stop; the current basic feasible solution is 


optimal. Otherwise, calculate y, = B™ lay. If y, <0, stop; the optimal objective 


value is unbounded. If y, < 0, insert the column EC =. Ck to the right of the 
k 


tableau as follows: 
BASIS INVERSE 


ear ae Ea) 


Determine the index r via the standard minimum ratio test: 


b, bj 
—= minima [Ve > 0 
Vik 


rk lsism 


Pivot at y,,. This updates the tableau. Repeat the main step. 


Example 5.1 
Minimize -x, - 2x7 + %3 — X%4 — 4x5 + 2Xx¢ 
subjectto x) + XxX. + %3 + Xy + Xy + Xe S 6 
2x -— XT 2x3 + X4 < 4 
Xy + Xg + 2X5 + X6 < 4 
2 0. 


> X25 X3,  X4> ase *6 
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Introduce the slack variables x7, xg, and x9. The initial basis is B = [a7, ag, a9] = 


1,. Also, w = ¢gB"! = (0, 0, 0) and b =b. 


Iteration 1 


BASIS INVERSE RHS 


Here, w = (0, 0, 0). Noting that z;-c; = wa;— c,;, we get 
z-¢, =1, Z2—C2 =2. 23-03 =-] 
Z4—C4 =], Z5 —C5 =4, 26 —% =~2, 


Therefore, k= 5 and x5 enters the basis: 


10 07} 1 1 
ys = Bas =/0 1 O}} 0) =] 0). 
0 0 1i/2 2 


4 
[2=*]- 1 
Ys 


to the right of the tableau and pivot at y35 = 2. 


Insert the vector 


Iteration 2 


Now, w = (0, 0, —2). Noting that z; - c; = wa; — c;, we get 
z-q =1, 22-2 = 2, 73 -¢3 = —3, 


7 -¢4 = —l, 2 - C6 = —4, 
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Zg — Cg = —2. 


Therefore, k = 2 and x enters the basis: 


10 -/21f 1 1 
y,=B'a,=/0 1 Of/-t}=]-1]. 
0 0 1/2]; 0 0 
Insert the vector 
Cane 
~{—4 
Y2 0 


to the right of the tableau and pivot at y,>. 


BASIS INVERSE RHS X 
z 
*7 ) 
Xe - 
Xs 0 
Zz 
% 
Xg 
X5 
Iteration 3 
We now have w = (~2, 0, -1). Noting that z; — c; = wa; — c;, we get 


Zyz-cy=-l, 23-c3=-4, z4-c4 =-2, 

2-4 =-5, Z9—-Cg=—l. 
Since z; — c; <0 for all nonbasic variables (x7 just left the basis and so z7 ~c7 < 
0), we stop; the basic feasible solution of the foregoing tableau is optimal. 


Comparison Between the Simplex and the Revised Simplex Methods 


It may be helpful to give a brief comparison between the simplex and the revised 
simplex methods. For the revised method, we need an (m + 1) x (m + 1) array as 
opposed to an (m+ 1) x (7 + 1) array for the simplex method. If 7 is significantly 
larger than m, this would result in a substantial saving in computer core storage. 
The number of multiplications (division is considered a multiplication) and 
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additions (subtraction is considered an addition) per iteration of both procedures 
are given in Table 5.1. In Exercise 5.6 we ask the reader to verify the validity of 
the entries of the table. 

From Table 5.1 we see that the number of operations required during an 
iteration of the simplex method is slightly less than those required for the revised 
simplex method. Note, however, that most practical problems are sparse, that is, 
the density d of nonzero elements (number of nonzero elements divided by the total 
number of elements) in the constraint matrix is usually small (in many cases d < 
0.05). The revised simplex method can take advantage of this situation while 


calculating z; — c;. Note that z; = wa, and we can skip zero elements of a; 


while performing the calculation wa; = Be Way Therefore, the number of 


operations in the revised simplex method for calculating the (z; — c; )-values (this 


is called the pricing step) is given by d times the entries of Table 5.1, substantially 
reducing the total number of operations. (At the end of this section, we also discuss 
a partial pricing strategy whereby the (z; — c;)-values are computed at each 


iteration for only a selected set of nonbasic variables, thus considerably saving on 
computational effort.) While pivoting, for both the simplex and the revised simplex 
methods, no operations are skipped because the current tableaux usually fill 
quickly with nonzero entries, even if the original constraint matrix was sparse. 
(However, see the section on Implementation Remarks and the Notes and 
References section on preserving sparsity in a factored—form implementation of the 
revised simplex method.) 

Empirically, it is often suggested that on the average in most instances, 
the simplex method requires roughly on the order of m to 3m iterations. (However, 
it has also been suggested that the number of iterations is often proportional to ” or 
even log, n.) Examining Table 5.1, since the number of operations per iteration is 


of order O(mn) (that is, it is bounded from above by some constant times mn), the 
average empirical complexity of the simplex method is O(m?n). However, 


sparsity is usually present and is exploited in simplex implementations. This is 
done by storing data in packed form (in which only nonzeros are stored with 
appropriate pointers and are alone used in arithmetic calculations), as well as 


Table 5.1. Comparison of the Simplex and the Revised Simplex Methods 


OPERATION 
METHOD PIVOTING PRICING TOTAL 


Multipli~ | (m+ 1)\(#—m+1) mu-—m)t+nt+) 
cations 


| Additions | min—m+1) | = | min-m+t) 


Multipli~ 2 m(n — m) 2 
: + —m)+(m+ 
m 


Additions | m(m + 1) 


Simplex 


Revised 
Simplex 
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by maintaining the basis in factored form, as we shall see shortly. Because of this, 
a regression equation of the form Km“ nd’, where a@ = 1.25 -2.5 and B = 0.33, 


usually provides a better fit for overall computational effort. 

To summarize, if n is significantly larger than m and if the density d is 
small, the computational effort of the revised simplex method is significantly 
smaller than that of the simplex method. Also, in the revised simplex method, the 
use of the original data for calculating the (z 7 7 ¢;)-values (pricing) and the 


updated column y; tends to reduce the cumulative round-off error. 


Product Form of the Inverse 


We now discuss another implementation of the revised simplex method where the 
inverse of the basis is stored as the product of elementary matrices. (An elementary 
matrix is a square matrix that differs from the identity in only one row or one 
column.) For sparse problems, this leads to fewer storage and computational 
burdens, and it provides greater numerical stability by reducing accumulated 
round—off errors. Although we describe next a more efficient alternative 
implementation scheme based on an LU factorization of the basis that is popularly 
used in practice, the present discussion is relevant because it lays the conceptual 
foundation of several important constructs and ideas. 

Consider a basis B composed of the columns ag,, 4g, ,....g and sup- 


pose that B | is known. Now, suppose that the nonbasic column a; replaces a B,> 


resulting in the new basis B. We wish to find B! interms of B!, Noting that a, 
= By, and ag = Be; where e; is a vector of zeros except for | at the ith position, 
we have 


A 


B 


(Ap AB) AR ARAB > AB,) 
(Be, Be,,..., Be,_,, By,, Be,.),..., Be,,) 
= BT, 
where T is the identity with the 7th column replaced by y,. The inverse of T, 


I 


which we shall denote by E, is given as follows: 


7th column 


0 |< rthrow. 
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Therefore, Bo! = T 'B"! = EB”! where the elementary matrix E is specified 
above. To summarize, the basis inverse at a new iteration can be obtained by 
premultiplying the basis inverse at the previous iteration by an elementary matrix E. 
Needless to say, only the nonidentity column g, known as the efa vector, and its 
position r need be stored to specify E. 

Let the basis B, at the first iteration be the identity I. Then the basis 


inverse B;! at Iteration 2 is By! = E|By! = E|I= E,, where E, is the elementary 


matrix corresponding to the first iteration. Similarly, Bj! = E,Bz! = E,E,, and 
in general, 

B; = E,_)E,_)---E,E). (5.1) 
Equation (5.1), which specifies the basis inverse as the product of elementary 
matrices through their associated eta vectors, is called the product form of the 
inverse. Using this form, all the steps of the simplex method can be performed 


without pivoting. First, it will be helpful to elaborate on multiplying a vector by an 
elementary matrix. 


POSTMULTIPLYING 


Let E be an elementary matrix with its nonidentity column g appearing at the rth 
position. Let ¢ be a row vector. Then 


position r 
cE = (Cy 5C25+-45Cp) PS 82 ul 
00 + g, 1 (5.2) 


m 
Coane x eineraivnen 


i=] 
(Cp, Cy 50005 Cp_y 5 CBs Cp] o-ee9 Cy) 


In other words, cE is equal to ¢ except that the rth component is replaced 


by cg. 
PREMULTIPLYING 
Let a be an m—vector. Then 
1 81 0 ay 
Ea = |0 Z, Olla 
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Bt B19, 4) 8\ 
= &r4y am 0 +a, &r 
An + Zm4y Ay Zn 
In other words, 
Ea=a+a,g (5.3) 


where 4 is equal to a except that the th component a, is replaced by zero. 


With the foregoing formulas for postmultiplying and premultiplying a 
vector by an elementary matrix, the revised simplex method can be executed 
without pivoting. The following discussion elaborates on the simplex calculations. 


COMPUTING THE SIMPLEX MULTIPLIER VECTOR w= zB! 


At iteration ¢ we wish to calculate the vector w. Note that 

w=cpB,! =cpE,_)E,7---E)E). 
Computing w can be iteratively performed as follows. First compute ¢gE,_, 
according to Equation (5.2). Then apply Equation (5.2) to calculate (egE,_;)E,_>, 
and so forth. This backward transformation process is sometimes referred to as a 


BTRAN process. After w is computed, we can calculate z prep = Wa; cj for the 


nonbasic variables. From this we either stop or decide to introduce a nonbasic 
variable x;. 


COMPUTING THE UPDATED COLUMN y,; AND THE RIGHT-HAND- 
SIDE b 
If x, is to enter the basis at iteration 4, then y, is calculated as follows: 

yx = By a, =E,)E,_9--E2E ay. 
This computation can be executed by successively applying Equation (5.3) in the 
order E),E5,...,E,_;. This forward transformation process is sometimes referred 
to as an FTRAN process. If y, <0, we stop with the conclusion that the optimal 
objective value is unbounded. Otherwise, the usual minimum ratio test determines 
the index r of the variable xg that leaves the basis. Thus, x, enters and xg_ leaves 
the basis. A new elementary matrix E, is generated where the nonidentity column 
g is given by: 

Vik Vrk 


Wyy4 


—Vmk Vrk 
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and appears at position 7. The new right—hand-side 1s given by 
B;b =E,B;'b. 


Since B,'b is known from the last iteration, then a single application of Equation 


(5.3) updates the right-hand-side vector b. 
UPDATING THE BASIS INVERSE 


The basis inverse is updated by generating E,, as discussed previously. It is 
worthwhile to note that the number of elementary matrices required to represent 
the basis inverse increases by | at each iteration. If this number becomes large, it 
would be necessary to reinvert the basis and represent it as the product of m 
elementary matrices (see Exercise 5.9). It is emphasized that each elementary 
matrix is completely described by its nonidentity column and its position. 


Therefore, an elementary matrix E could be stored as H where g is the 


nonidentity column and + is its position. 


Example 5.2 
Minimize -xy} — 2x7 + x 
subjectto x + Xx + x3 5 4 
—X, + 2x9 - 2x3 S 6 
2x, + XQ <5 
Xx, Xo, x 2 0. 


Introduce the slack variables x4, x5, and xg. The original basis consists of x4, xs, 
and X6- 


Iteration 1 


XB, X4 4 x] 0 
Xp =| Xp, [=| 5 |=| 6], Xv =| xX. |=] 0], 
XR, X6 5 x3 0 


z=0, 
w=cp =(0, 0, 0). 
Note that z eS WES = Ce Therefore, 
z-q=hL Z2—¢€) = 2, 23 -¢; =—l. 


Thus, = 2 and x, enters the basis: 
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1 

Y2 =a) =| 2 

| 

Here, x B, leaves the basis where r is determined by 


minimum {22,2 = minimum {2.5.3} =3. 
M2 y22 32 121 


Therefore, r = 2, that is, xg, = x5 leaves the basis and x2 enters the basis. The 


nonidentity column of E, is given by 


—yi2/¥22 | [-1/2 
g= Wyo |=] 1/2 
—¥32/V22 =2 
and E, is represented by HE 


Iteration 2 


Update b . Noting Equation (5.3), we have 


b=E,|6|=|0]+6) 1/2]/=/31], 
5| {5} |-1/2] |2 
XB X4 1 x 0 
Xp =| Xp, -[3|-[3} XW -|2|-(3) 
xp, | Le] 12 x,| [0 


z=0-b,(z) -c)) =-6, 

w =CczE, = (0,-2,0)E). 
Noting Equation (5.2), then w = (0, —1, 0). Note that zj;—€;= wa; —¢;. 
Therefore, 

Zy—-¢) =2, 23-c3=1. 


Thus, k= 1 and x, enters the basis. Noting Equation (5.3), 


1) (1) [1/2] F 3/2 
y, =Eya, =E,|-1/=/0]-| 1/2|=| -1/2]. 
FUE Aaes 


Then, XB. leaves the basis where r is determined by 


7 (2 a 3 | Lb | 2 
minimum 4 ——,—— 7 = minimum, ——-,——- p=. 
Yi 31 3/2° 5/2). =3 
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Therefore, r = 1, that is, xg = x4 leaves and x enters the basis. The nonidentity 


column of E, is given by 
Vy 2/3 
Z=|-yo/vi {=| 1/3}. 
-yx/Mi} L-5/3 
Also, E, is represented by i 


Iteration 3 


Update b. Noting Equation (5.3), we have 


1 0 2/3 2/3 
b=E,/3/=/|3|4+1) 1/3 |=] 10/3}, 
2 2 -5/3 1/3 
XB, x 2/3 x4] [0 
XB XB, =| X%2 |= 10/3 . XN =| %5 |= 0 ; 
X6 1/3 x3] [0 


XB, 
z= -6-B(z, ~c) =~ 22/3, 
w =cpE FE, =(-1,-2,0)E,E). 
Applying Equation (5.2) twice, we get 
cgE, = (-4/3, -2,0), 
Ww =(CgE> )E, = (-4/3,-1/3,0). 
Note that Zz; —C; = WA; —C;. Therefore, 


J : | 
23 - C3 =-5/3, 25 — C5 =-1/3. 
Since z;—c; <0 for all nonbasic variables, the optimal solution is at hand. The 


objective value equals —22/3 and 
(2X1 ,X9,X3,%4,5X5,.%6) = (2/3, 10/3,0,0,0, 1/3). 


LU Decomposition or Factorization of the Basis 


The product form of the inverse essentially records the history of pivots per formed 
through a collection of the eta vectors. Although this method is conceptually 
important and useful, it is computationally somewhat obsolete because it is based 
on a complete Gauss—Jordan elimination technique for solving systems of 
equations. A more popular technique used by computer packages is the LU 
factorization method, which is based on the more efficient Gaussian 
triangularization strategy. It derives its name from its use of lower and upper 
triangular factors of the basis B. This method is most useful when the problem is 
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large-scale and sparse, and it is accurate and numerically stable (round—off errors 
are controlled and do not tend to accumulate). 

In implementing the simplex algorithm, the systems of equations that 
need to be solved are Bx, = b and wB = Cp, and then By, = a, once an entering 
variable x; is determined. The first of these equations gives the basic variable 
values. Since the variable values are simply updated at every iteration, it only 
needs to be solved initially or whenever B is periodically refactored from scratch in 
order to maintain numerical accuracy. The second system computes the simplex 
multiplier vector w, which is used to price the nonbasic variables via the 


relationship z;—c; = wa; —c,. Once a variable x, having z, — cy > 0 has been 


chosen to enter the basis, its updated column y, needs to be computed. This is 
done via the third system of equations. The vector y; provides the information on 
how the basic variables change when x, is increased and determines the exiting 
variable via the minimum ratio test. Hence, a new basis is obtained along with the 
(possibly) revised basic variable values. This process is then repeated. 

Now, if B happens to be an upper triangular matrix, say, then the 
foregoing systems of equations can be solved easily and accurately. The systems 
Bx, = b and By, = a, can be solved by a backward substitution process in which 
the last component of the solution vector is given directly by the last equation. This 
is substituted in the next to last equation to obtain the next to last component of the 
solution vector, and so on. The system wB = cz can be solved similarly by a 


forward substitution process in which the components of w are obtained 
sequentially in the order wj, Wy, ... Wy. 


On the other hand, if B is not (upper) triangular, then it can be made so 
through a series of row operations. Algebraically, this process can be represented 
as a premultiplication of B by a nonsingular, row—operation transformation matrix 
R such that RB = U, an upper triangular matrix. If this is done, then to solve the 
system By; = a;, we can premultiply both sides by R to get RBy, = Ra;, that is, 


Uy, = a,, where a, = Ra,;. Hence, the upper triangular system Uy, = a; can 
now be solved by a backward substitution process. An identical strategy can be 
used to solve the system Bxz = b. In order to solve the system wB = cp, consider 


the affine transformation w = w’R. Then the foregoing system becomes w’RB = 
Cp or wU = cp, from which w’ can be obtained via a forward substitution 


process. Knowing w’, we can now compute w as wR. Hence, if the row 
operations involved are not too computationally intensive and are well conditioned 
(do not involve divisions by small magnitude numbers, for instance), then the 
systems of equations can again be solved conveniently and accurately. 

Observe that if the row operations required to upper triangularize B 
involve a simple row by row Gaussian reduction process, then R would be a lower 
triangular matrix. In other words, the first row of B would be left as is, the second 
row would have a multiple of the first row added to it in order to zero out the (2, 1) 
element of B, the third row of B would have a multiple of the first row and a 
multiple of the second row added to it in order to zero out the (3, 1) and the (3, 2) 
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elements, and so on. In this case, we would have B = R Us LU, a product of a 


lower triangular matrix L = R7! and an upper triangular matrix U. Hence, such a 
factorization is called an LU factorization, even in instances when R is not lower 
triangular. 


Obtaining the LU Factorization of B 


Typically, in addition to the Gaussian pivots or reductions, we also need to perform 
permutations of rows in the factorization in order to obtain desirable pivot 
elements. For example, to begin, we may wish to make that row the first row that 
has the largest absolute valued coefficient in the first column, so that the Gaussian 
pivots associated with zeroing out the first column under the (1, 1) element are 
numerically well conditioned. (Of course, if the starting (1, 1) element in B is zero, 
then some such permutation is imperative.) Hence, the row operation matrix R 
may involve a (row interchange) permutation matrix P, followed by a Gaussian 


reduction or pivot matrix G, used to triangularize the first column (or the first 
nontriangular column), then possibly another permutation matrix P, followed by a 
Gaussian triangularization matrix G,, and so on. Hence, if we use r such 
operations, we will have 

R =(G,P,)---(GyP2)(G)P)). 


Accordingly, a forward transformation process called FTRAN is used to compute 
aj, = Ra, while solving the system By, = a, by employing the operators (G;P;) 
in the order 1,....7 on az. Similarly, a backward transformation process called 
BTRAN is used to compute w = w’R while solving the system wB = cz by 
postoperatively using G;P; on w’ in the order r,r—1,..., 1. 


Now, let us discuss the construction of the operators defining R. Suppose 
that so far we have upper triangularized columns 1,...,k — 1 of B, and we are now 
considering the triangularization of a nontriangular column & using, for example, 
the operator G;P;. Let B,; be the current partially triangular matrix. Then we first 
determine the largest absolute valued element in rows k, k + 1, ..., m for column k 
in B,. If this happens to be the (4, k) element itself, then no row interchange 
(permutation) is necessary, and in this case P; = I. Otherwise, if this happens to be 


the element (¢, k), then we need to interchange rows k and ¢ Hence, the 
permutation matrix that accomplishes this is given as follows, where the elements 
not shown are zeros: 
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(5.4) 


Having done this, suppose that the matrix B,; now appears as follows, where the 
blank section has all zeros: 
k 


|=P,(G,_,P.1)(G,P,)B. (5.5) 


Now, in order to zero out the elements in column & and in rows (k + 1), ..., m, we 
need to premultiply B, by the following Gaussian pivot matrix G;, where again, 
all elements not shown are zeros: 


k 


G, =k 1 . (5.6) 


bask Oe 


—Bange | Dia l 
Observing G;, it is evident why we would not want 5,, to be too small. 
Also, compare G; with the Gauss—Jordan pivot or elementary matrix E used in the 
product form of the inverse. Needless to say, the matrices P; and G; are stored in 
compact or packed form. Namely, only the relevant permutation performed by P; 
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is stored, and similarly from G;, only the nonzeros in rows k + 1, ..., m in column 
k, along with their positions are stored. This operation now produces G;B, = 
(G,P;)---(G,P,)B from Equation (5.5). This process may now be continued until 
we obtain RB = (G_P..)---(G,P, )B =U. 


Example 5.3 


Suppose that the following matrix B needs to be factored and we need to solve the 
system Bxz = band wB= cp, where 


t 3. 3 9 
B=|2 1-1], b=|/3|, eg, =(11,2) 
(99 q 


Beginning with column 1, we first perform an interchange of rows | and 2 in order 
to make the (1, 1) element the largest (in absolute value) in column 1. This 
corresponds to the permutation matrix P,;, which then yields the matrix B). 
; 1 } F 1 7 
P=/1 0 Of, B,=PB=|1 3. 2). 
00 1 be? $2 


Next, we triangularize column | of B, via the Gaussian pivot matrix G,, where 


10 0 
G, =|-1/2 1 0], 
=) 3: 0. il 


which gives, 
2 1 -l 
(G,P,)B=|0 5/2 5/2). 
6: 3/2. 5/2 
Continuing, we can now take P, as the identity matrix, which gives B, = 
P,(G,P,)B =(G,P,)B as before. We then get from Equation (5.6): 


1 0 0 
G, =|0 1 0}, 
0 -3/5 1 
which gives 
: ro a | 
(G>P))(G,P,)B=|0 5/2 5/2}=U. 
a) 1 


Hence, R= (G,P,)(G,P,) and U is as previously given. 
Now, to solve Bxp = b, we first compute Rb as follows using an FTRAN 
process: 
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3 3 3 3 
Rb = (G>P>)(G,P,)b = (GP,)G,| 9 | =(GyP,)| 15/2 |= Go} 15/2 | =| 15/2 |. 
7 11/2 11/2 I 


Therefore, since RBxz = Rb, that is, Uxg = Rb, where U is as given above, we 


compute Xp = (1,2,1)' by backward substitution. 

Similarly, in order to solve WB = cp, we first solve w'U = cz by forward 
substitution and then compute w = wR. This gives w’ = (1/2, 1/5, 2). 
Consequently, applying a BTRAN process, we obtain 

w = wR=(1/2,1/5,2)[(G2P2)(G)P))] 
= (1/2,-1,2)P)(G,P,) = (1/2, -1,2(G,P,) 
(0,-1,2)P, =(-1,0,2). 


The reader may wish to observe how the foregoing calculations are performed with 
P, and G; stored in the previously mentioned compact form. 


Updating the LU Factors 


Suppose that we have selected x, as the entering variable and have determined x, 


as the leaving variable in the usual manner. The task now is to update R so that 
when operated on the new basis, it produces an upper triangular matrix. Toward 
this end, suppose that we delete from B the leaving column, say, column +, then 
move the columns (7 + 1), ..., m leftwards, that is, make the current column i as the 
new (i — 1)th column for i=(r + 1), ..., m, and insert the entering column a, of x, 


as the last (mth) column. Let this new basis be denoted by B,.y,. (We would need 


to accordingly permute the updated right~hand-side so that the new basic variable 
values appear in the same order as their columns are arranged in B,ey.) Noting 


that RB is an upper triangular matrix, we now have RB,.y appearing as shown 


below, where the blank section has all zeros: 
r m 


(5.7) 
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Note that the columns 1, ....7— 1 of RB, .y are as in U; the columns r, 
.., m— 1 of RB,ew are precisely the columns (7 + 1), ..., m of U, and the final 
column in RB,ew is Ra;. Note that Ra, is already available from the solution of 
the system By, = a; (why?). Hence, we delete column r from U, move the col- 
umns that are to the right of it one position leftwards, and insert Ra; as the last 
column in U, in order to obtain RB,.w. The type of matrix depicted in Equation 


(5.7) is called an upper Hessenberg matrix. Note that the elements in the black— 
shaded region in Equation (5.7) used to be along the diagonal of U, and hence are 
nonzeros, since U is nonsingular. Now, we can perform additional permutations 
and Gaussian pivots embodied in Equations (5.4) and (5.6) as before in order to 
triangularize RB,ey. The additional factors of the type (G;P;) used are appended 


to R in order to derive the new factor R. 
Example 5.4 


Consider Example 5.3. Suppose that we price the nonbasic variables and select 
some variable x4 as the entering variable. Let a, = (1,—1,0)' and C4 =—2. Hence, 
Z4 —C4 = Wag —C4 = (-1,0,2)(1,-1,0)' — (-2) = 1 > 0. Furthermore, the system 
By, = a4 gives RBy, = Ra,, where RB = U, and where 


-l 
Ra, =(G5P) )(G,P, ag =(G2Py)G,| | 
F anes (5.8) 
=(G>P)) 3/2 =G, 3/2 = 3/2 : 
1/2 /2| |-2/5 


Hence, Uy, = Rag gives y, = (—6/5,1,—2/5)’. Consequently, performing the 
minimum ratio test, the basic variable corresponding to the second column of B 
leaves the basis. Eliminating this second column of B and appending ay as the 
final column of B results in the new basis B,.y. From U and Equation (5.8), we 
get (as in Equation (5.7)): 


a | 
RByew =|0 5/2 3/2]. 
G5 1 205 


Furthermore, on pivoting, the right-hand-side updates to (17/5,2,9/5)'. Because 
of the rearrangement of the columns of Bey, the new right-hand-side is 


(17/5,9/5,2)'. To triangularize the second column of RByew, We use P; = I and, 
from Equation (5.6), we use 
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| 0 0 
G; =|0 1 0}, 
0 -2/5 1 
which gives, 
2 -1 -l 
(G3P3 (GzP. )(G,P, Brew = (G3P3)RBpew =| 0 5/2 3/2 =Unew- 
0 0 -l 


Hence, the new factors R and U are, respectively, Rpew = (G3P3)(G2P2)(G)P;) 
and Unew 
w, and price the nonbasic variables, and then continue. 


as given previously. We now compute the new simplex multiplier vector 


Some Implementation Remarks 


It should be evident from the foregoing discussion that the LU factorization 
technique is particularly well suited for sparse problems, and it can benefit greatly 
if the factors R and U are themselves sparse. Toward this end, one can initially try 
to permute the rows and columns of the (sparse) matrix B to make it as “upper 
triangular’ as possible, while trying to reduce any “bumps” and “spikes” that 
protrude below the diagonal. An attempt is also usually made to preserve sparsity 


in the factors in this fashion, so that although B! may tend to fill up (become 
dense relative to B), the number of nonzeros in the Gaussian pivot matrices and in 
U tend to be of the same order as the number of nonzeros in B itself. Of course, in 
order to maintain accuracy and sparse storage requirements, and to reduce 
computational effort, the current basis should be periodically refactored from 
scratch. Exercise 5.59 and the Notes and References section provide further 
implementation guidelines. 

Most problems also benefit by scaling, in which both rows and columns 
are sequentially scaled by dividing throughout with, for example, the average 
magnitude of the nonzeros in order to make the coefficient magnitudes in the rows 
and columns compatible with each other. This enhances the numerical accuracy of 
the algorithm and can also dramatically reduce the solution effort by virtue of the 
simplex path generated. 

Motivated by the same concepts is the use of a steepest edge entering 
variable selection strategy. Here, instead of selecting the variable having simply 
the most positive (z ; i ‘j)~value to enter the basis (Dantzig’s Rule), we base the 


. sigs = 2 
selection on the most positive value of (2; - c;)/y;, where y; = ltd ays 
Note that y ; is the Euclidean norm of the direction vector along the edge associated 
with increasing the nonbasic variable x; by a unit. Hence, (z pe Wy; is the 


negative rate of change in the objective function value per unit distance along this 
direction. Of course, in order to conserve solution effort per iteration, the y ;— 


quantities need to be updated rather than recomputed from one iteration to the next 
(see the Notes and References section). This rule significantly enhances the 
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performance of the simplex algorithm over the use of Dantzig’s Rule, particularly 
on unscaled problems. (It has also been observed to assist more strongly with the 
dual simplex method that is discussed in Chapter 6.) 

Another related strategy is partial pricing or suboptimization. Thus far, we 
have conducted a total pricing of all the nonbasic variables at each iteration. This is 
convenient and advantageous for problems having no more than 1000 or so variables. 
However, for larger-sized problems, such a total pricing operation can be 
computationally expensive. It is usually preferable to perform a partial pricing or a so— 
called suboptimization process. We describe here one particular implementation of this 
process using Dantzig’s Rule for selecting an entering variable. To begin, all the 
nonbasic variables are priced and the largest (z ; —c ; )—value, say, z, — cy = Amax > 9, 


is found. A list Z is constructed of the current basic variables and those nonbasic 
variables for which z; —c; lies within some p, percent of Ajax. At this point, the 


variables not in Z are ignored, and the iterations continue by selecting the nonbasic 
variable having the most positive (z ; —c ; )—value in L for entering the basis. However, 


once the most positive (z ; — ‘j)-value falls below some pz percent of A or when 


max? 
the simplex method goes through some N consecutive degenerate iterations, the list Z is 
reconstructed. If this list were not reconstructed, then the algorithm would eventually 
solve the problem where the ignored nonbasic variables are permanently fixed at zero; 
hence the name “suboptimization” of this technique. Typical values of p, and pp are, 


respectively, 0.8 and 0.67 for sparse problems (d < 0.05 — 0.1) and 0.7 and 0.4 for 
dense problems. (Here, d is the density of the matrix A.) A typical value of N is about 
10-30. 

Finally, we mention that for large-scale problems, it is useful to have a 
matrix generator program in order to automatically generate the constraints and the 
objective function based on the structure of the application instead of manually 
inputting the data. The software would then store the nonzero coefficients in 
packed form as noted earlier, and then use them appropriately to perform the 
foregoing computations. Many modeling languages are now available as user— 
friendly interfaces to enable the specifications of problems in algebraic symbolic 
format, rather than require a laborious construction of nonzero matrix coefficients 
in specified packed formats. The interested reader is referred to the Notes and 
References section for further discussions on these topics. 


5.2 THE SIMPLEX METHOD FOR BOUNDED VARIABLES 


In most practical problems the variables are usually bounded. A typical variable x; 
is bounded from below by £ j and from above by u jo Where £; < u;. If we 


denote the lower and upper bound vectors by @ and u, respectively, we get the 
following linear program with bounded variables: 

Minimize cx 

subjectto Ax=b 


£<x<u. 
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By standard transformations, we can assume without loss of generality that @ ; is 


finite for each j = 1, ..., n (why?). If 2 = 0, the usual nonnegativity restrictions are 
obtained. In fact, any lower bound vector can be transformed into the zero vector 
by using the change of variables x’ = x — £. The most straightforward (and the 
least efficient) method of handling the constraints 2 < x < u is to introduce the 
slack vectors x; and x, leading to the constraints x + x; =u and x — x, = &. 
This increases the number of equality constraints from m to m + 2n and the number 
of variables from 7 to 3n. Even if 2 = 0 or is transformed into 0, as discussed, the 
slack vector x; is needed, which increases both the constraints and variables by n. 

From the foregoing discussion it is clear that the problem size (and hence 
the computational effort) would increase significantly if the constraints £<x <u 
are treated in the usual manner by introducing slack vectors. The simplex method 
with bounded variables handles these constraints implicitly in a fashion similar to 
that used by the simplex method to handle the constraints x > 0. Therefore, the so— 
called “working basis” remains of size m x m, and hence such a method is known 
as a compact basis method. As in the simplex method, the algorithm of this section 
moves from a basic feasible solution to an improved basic feasible solution of the 
system Ax = b, @ < x < u (at a nondegenerate pivot step), until optimality is 
reached or unboundedness is verified. 

Now, observe that the feasible region of the foregoing problem is a 


polyhedron in R”, and as before, its extreme points are feasible points at which 


some n linearly independent hyperplanes are binding. Since Ax = b gives m 
linearly independent hyperplanes that are binding at each feasible solution, we seek 
p =n7— m appropriate additional binding hyperplanes from the inequalities £< x < 
u in order to make up the required set of n linearly independent binding equations 
for defining each extreme point solution. (Throughout we only consider constraints 
having finite upper or lower bounds to be binding.) Consequently, there are p = n — 
m degrees of freedom remaining after imposing the constraints Ax = b, and so, the 


given polyhedron is embedded in R?. We therefore need to associate with an 


extreme point some p (independent) nonbasic variables fixed either at their lower 
or their upper bounds such that the remaining m (dependent) basic variables are 
uniquely determined via the system Ax = b. If more than p, say, g, of the 
inequalities 2 < x < u are binding at an extreme point, then such an extreme point is 
degenerate and has q — p = | of the associated basic variables at one of their 
bounds. Hence, we define a basic feasible solution as follows, so that a solution x 
is an extreme point if and only if it is a basic feasible solution. Details of the proof 
based on this exposition are very similar to the case of the set X = {x: Ax = b, x > 
0} and are left to the reader in Exercise 5.17. 


Definition (Basic Feasible Solutions) 


Consider the system Ax = b and @ < x < u, where A is an m x matrix of rank m. 
The solution X to the equations Ax = b is a basic solution of this system if A can be 
partitioned into [B, N,, N2], where the (square) matrix B has rank m, such that with 
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x partitioned accordingly as (xg, Xy,, Xy,) we have Xx, = £y,, Xy, = Uy,» and 
: = _ pl -1 -l = 

accordingly, Xz; = Bb — B-N,éy,, ~ BNouy,. Furthermore, if £3 < Xg < 

ug, then xX is said to be a basic feasible solution. The matrix B is called the 


(working) basis, xg are the basic variables, and xy, and Xy, are the nonbasic 


variables at their lower and upper limits, respectively. If, in addition, 2p < Xp < 
uz, then X is called a nondegenerate basic feasible solution; otherwise, it is called 
a degenerate basic feasible solution. 

The partition [B, N,, N2] corresponding to a basic (feasible) solution is 
said to be a basic (feasible) partition. Two basic (feasible) partitions are said to be 
adjacent if all but one nonbasic variable coincides in the two partitions, being fixed 
at the same bounds. Note that for a degenerate extreme point, x, each choice of p 
= (nm — m) defining hyperplanes from the set 2 < x < u that, along with the m 
constraints Ax = b, yield » linearly independent hyperplanes binding at x, 
corresponds to a basic feasible partition associated with x. 


Example 5.5 
Consider the region given by 
xX + X% S 5 
xX, + 2X9 < 4 
< x, < 4 
-l < x2 < 4. 


First introduce the slack variables x3 and x4. This gives the following system 
(note that a close examination of the system shows that v3; and u4 can be replaced 
by 6 and 10, respectively, if so desired): 


x] + xy + x3 = 5 
—X} + 2x + x = 4 
< x, < 4 

-—l< x2 < 4 

Os x3 < © 

Os X4 < ow. 


We would like to find all the basic feasible solutions of this system. This can be 
accomplished by extracting a basis of the first two constraints, solving the basic 
variables in terms of the nonbasic variables, and then fixing the nonbasic variables 
at their lower or upper bounds. To illustrate the method select, say, a, and a4 as 


the basic vectors. Then 
1 0 = 1 
B=[ay.a4]=|) a and BE! -|3 i 


Multiplying the first two constraints by B! and transferring x, and x3 to the 
right—hand-side, we get 
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%y = 3S - NH - x3 
xX = 6 + 3x, + 2X3. 


Now, fix x, at its lower or upper bound, x; at its lower bound, and solve for x 


and x4: 


1. x, =0, 3% =0> x =5,and x,=~—6. Since x4 <0, this basic solution 
is not a basic feasible solution. 
2. x =4, x3 =0> x =I1, and x, = 6. Therefore, (x), x2, x3, %4) = 


(4, 1, 0, 6) is a basic feasible solution. 


The other basic solutions can be obtained in a similar manner. If all of the possible 
bases were enumerated, we would see that the basic feasible solutions are (2,3,0,0), 
(0,2,3,0), (4,1,0,6), (0, -1,6,6), and (4, -1,2,10). Projecting these points in the (x, 
X)) space, we get the extreme points v,, .... Vs shown in Figure 5,1. In this 
example, n = 4, m = 2, and so p =n — m = 2. In fact, the representation shown in 
Figure 5.1 is in the p = 2-dimensional space corresponding to x; and x» as the 
independent variables. Note how each extreme point is formed by some p = 2 
linearly independent defining hyperplanes binding in this space. In particular, (x3 = 
£3, Xg= £4) gives vj, (x = £1, X4= £4) gives V2, (x = uy, X3 = £3) gives v3, (x 
= £), x)= €) gives v4, and (x; = m, xX) = >) gives v5. Hence, as noted 
previously, the basic feasible solutions and the extreme points coincide. Observe 
also that the extreme points v4 and vs have the same working basis associated 
with them, namely, the columns of the variables x3 and x4. However, they differ 
in the bound values assigned to the (independent) nonbasic variables x, and x», and 


hence produce distinct basic feasible partitions and distinct basic feasible solutions. 
Furthermore, since all extreme points are nondegenerate here, each has a unique 
basic feasible partition associated with it, and the adjacent basic feasible partitions 
correspond to adjacent extreme points. We ask the reader to explore a 
generalization of this in Exercise 5.18 by imitating the analysis for the 
nonnegatively constrained case. 


Improving a Basic Feasible Solution 


We now know how to characterize a basic feasible solution. We also know that an 
optimal basic feasible solution exists provided that the feasible region is not empty 
and the optimum is finite (why?). Note, however, that the number of basic feasible 
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Figure 5.1. Basic feasible solutions. 
solutions is large. (The number of basic feasible solutions is bounded above by 


(ee For each possible way of extracting a basis there are 2”. ways of 


fixing each of the nonbasic variables at their lower and/or upper bounds.) 
Therefore, a systematic way of moving among the basic feasible solutions is 
needed. 

Now, suppose that we have a basis B and suppose that the nonbasic 


matrix is decomposed into N, and N>, that is, A = [B,N,,N.,]. Accordingly, the 
vector x is decomposed into [xg, xy,, Xy,] and ¢ is decomposed into [¢g, ¢y,, 
cy, ]. Both the basic variables and the objective function can be represented in 


terms of the independent (that is, nonbasic) vectors xy, and xy, as follows: 


xp = B'b-B'N)xy, -B 'Noxy, (5.9) 
Z = CgXp+Cy, Xn, +¢y, Xn, 

= cz(B 'b-B'N)xy, -B'Nyxy, )+en,Xy, +€n,Xy, 

= ¢gB 'b+(cy, ~¢gB 'N,)xy, + (cy, —¢gB 'N2)xy,- (5.10) 
Suppose that we have a current basic feasible solution where Xy, = £y,, Xv, = 
Uy,» and £g < Xg < ug. This solution is represented by the following tableau. 
The right—hand-side column gives the true values of z and xp (denoted by z and 
b, respectively) when x n= nN, and x Ny = Un, are substituted in Equations (5.9) 


and (5.10). We emphasize that this column does not give cpB |b and Bb, 
which are the right-hand-sides of Equations (5.10) and (5.9), respectively. 


Special Simplex Implementations and Optimality Conditions 225 


Now, we try to improve the objective by investigating the possibility of modifying 
the nonbasic variables. From Equation (5.10) and noting that ey, — ¢ pB™'N, and 


cy, — ¢ ,B'N, give the reduced costs c; ~ z; of the lower and upper bounded 


nonbasic variables, respectively, we get 


z=egB'b- ¥ (2;-c;)xj- D (2;-¢;)x (5.11) 
jeJ, jes 


where J is the set of indices of nonbasic variables at their lower bounds and J, is 


the set of indices of nonbasic variables at their upper bounds. For j ¢ J, if z yey 


> 0, it would be to our benefit to increase x ij from its current value of @ fe 


Similarly, for 7 € J, if z preg 0, it would be to our benefit to decrease x 7; from 
its current value of w;. As in the simplex method, we shall modify the value of 


only one nonbasic variable while all other nonbasic variables are fixed, and hence 
obtain a movement along an edge of the underlying polyhedral set. The index k of 
this nonbasic variable is determined as follows. First, examine 


maximum maximum {z ; ~ c;} smaximum {¢; - z;} : 
JeJ, jeJ2 


If this maximum is positive, then let k be the index for which the maximum is 
achieved. If k € J,, then x, is increased from its current level of £;, and if k € 


Jy, then x, is decreased from its current level of w,. If the maximum is 


nonpositive, then z; —c,; <0 for all je J, and z; —c; 2 0 for all fed. 


Examining Equation (5.11), this indicates that the current solution is optimal. 

To summarize, given a basic feasible solution, if z peepee for all non- 
basic variables at their lower bounds, and if z 7 — cy 20 for all nonbasic variables 
at their upper bounds, then we stop with the conclusion that the current solution is 


optimal. Otherwise, we choose a nonbasic variable x, according to the foregoing 
rule. If x, is at its lower bound, then it is increased; otherwise, it is decreased. 
These two cases are discussed in detail next. 
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Increasing x, from its Current Level 2; 


Let x, = £;,+ A, where Aj, is the increase in x, (currently A, =0). Noting that 
all other nonbasic variables are fixed and that the current value of xp and z are b 
and 2, respectively, substituting x, = ¢,+ A, in Equations (5.9) and (5.11), or 
by the usual Oxg / Ox, and 0z/Ox, information, we get 


Slaps] 1 
z=Z+A;,[-(z, -c,)]- (5.13) 
Since z, —c, > 0 (why?), then from Equation (5.13), it is to our benefit to 
increase A; as much as possible. However, as A, increases, the basic variables 


are modified according to Equation (5.12). The increase in A, may be blocked as 
follows. 


1. ABASIC VARIABLE DROPS TO ITS LOWER BOUND 


Denote the value of A; at which a basic variable drops to its lower bound by 7. 
From Equation (5.12) we have £3 < Xz = b — y,A,. Therefore, y,A, < b - 
fp. If y, < 0, then A, can be made arbitrarily large without violating this 
inequality and so 7, = © (that is, no basic variable drops to its lower bound). 
Otherwise, 7, is given by the following minimum ratio: 


b, - £3 b, -¢ 
minimum 4——* Viz > OP = er if y, £9 
= Vrk (5.14) 


The basic variable that reaches its lower bound is a candidate for xg . 


2. A BASIC VARIABLE REACHES ITS UPPER BOUND 


Denote the value of A, at which a basic variable reaches its upper bound by 72. 


From Equation (5.12), b — y,A, = Xp < ug, and hence —y,A, < ug— b. If 
y,; 29, then A, can be made arbitrarily large without violating this inequality, and 
sO 77 = © (that is, no basic variable reaches its upper bound). Otherwise, 7> is 
given by the following minimum ratio: 


Special Simplex Implementations and Optimality Conditions 227 


Vik —Yrk (5.15) 


ug, 5; up —5 
sn zs =: Vp <i peed ify, 20 


The basic variable that reaches its upper bound is a candidate for xB. 


3. x, ITSELF REACHES ITS UPPER BOUND 


The value of A; at which x; reaches its upper bound, w;,, is obviously uz, — £,. 
These three cases give the maximum increase in A; before being blocked 
by a basic variable or by x, itself. Obviously A, is given by 


A, = minimum{y,, 72, ug — fy}. (5.16) 


If A, =, then the increase in x; is not blocked, and by Equation (5.13), the 
optimal objective value is unbounded. If, on the other hand, A, <0, a new basic 
feasible solution is obtained where x, = &, + Aj, and the basic variables are 
modified according to Equation (5.12). 


Updating the Tableau When the Nonbasic Variable Increases 


The current tableau must be updated to reflect the new basic feasible solution. If 
Ay = uz — £,, then no change of the working basis is made and x, is still 


nonbasic, except this time it is nonbasic at its upper bound. Only the RHS column 
is changed to reflect the new value of the objective function and the new values of 
the basic variables. According to Equations (5.13) and (5.12), Z is replaced by 2 — 


(Zz, — cy) Ay and b is replaced by b- y,A;,. On the other hand, if A; is given 
by 7, Or 72, then x, enters the basis and xg leaves the basis, where the index r is 
determined according to Equation (5.14) if A, = 7, or according to (5.15) if A, = 
v2. Except for the RHS column, the tableau is updated by pivoting at y,;. Note 
that y,,; may be either positive or negative. Since the right-hand-side is computed 


separately, this should cause no alarm. The right~hand—side column is updated 
according to Equations (5.13) and (5.12), except that the rth component of the new 
vector b is replaced by @ ; + Aj to reflect the value of x;, which has just entered 
the basis. 

Alternatively, the right-hand-side vector can be updated directly with the 
rest of the tableau (although this is not the method of choice in practice). This, 
however, requires three distinct operations (which may be performed in any order). 
First, we multiply the nonbasic entering column by its current value (either £, or 


u,;) and add the result to the RHS vector. Next, we multiply the basic leaving 
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column by the value it will assume (either @ B, OF Up. ) and subtract the result from 


the RHS. Finally, we perform a normal pivot operation on the adjusted RHS 
vector. 


Decreasing x, from its Current Level u, 


This case is very similar to that of increasing x, and is discussed only briefly. In 
this case, z, — c, <0 and x, =u, — A, where A, > 0 denotes the decrease in 
x,. Noting Equations (5.9) and (5.11), we get 


xg |_| b Yk 
Papp (5.17) 
z=Z+A;,[z, —cz]. (5.18) 


The maximum value of A, is given by Equation (5.16) where 7, and 72 are 
specified below: 


—Yrk (5.19) 
eo if y; > 0; 


b, - £3 b, -£ 
sin a: Vik co) =i ify, 20 
y 


- up, ~ 5; Up. ~b, 
minimum Vip > OP = ify, £0 
¥2 =) \sism Vik Vrk (5.20) 


ioe) if y; <0. 


If A, =, then the decrease of x, is not blocked, and by Equation (5.18), the 
optimal objective value is unbounded. If A, <0, then a new basic feasible solution 
is obtained where x, = u, — A,, and the basic variables are modified according 
to Equation (5.17). 


Updating the Tableau When the Nonbasic Variable Decreases 


If A, = u, — £,, then x, is still nonbasic, but at its lower bound. The tableau is 
unchanged except for the RHS column, which is updated according to Equations 
(5.18) and (5.17). If Ay is given by 7; or 72, then x, enters the basis, and xg, 
leaves the basis where r is determined by Equation (5.19) if A, = 7% and by 
Equation (5.20) if A, = 72. Except for the RHS, the tableau is updated by pivoting 
at y,,. Again y,, could be either positive or negative. The RHS column is 
updated according to Equations (5.18) and (5.17), except that the rth component of 
the new vector b is replaced by uz — Ag to reflect the value of x, that has just 
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entered the basis. We may also again utilize the alternative method described 
previously to update the RHS vector. 


Getting Started 


If no basic feasible solution is conveniently available, we may start the lower— 
upper bounded simplex method with artificial variables. This is accomplished by: 
(1) setting all of the original variables to one of their bounds; (2) adjusting the 
RHS values accordingly; (3) multiplying rows, as necessary, by —1 to get b >0, 
and (4) adding artificial columns. The two-phase or the big-M method may be 
employed to drive the artificial variables out of the basis. 


Finite Convergence: Degeneracy and Cycling 


In the absence of degeneracy, at any iteration, the procedure described previously 
either moves from one basic feasible solution to an improved basic feasible 
solution, or declares unboundedness. Hence, it must terminate finitely, either with 
an optimal solution or with an indication of unboundedness. However, in the 
presence of degeneracy, it is possible to perform a degenerate pivot in which 7, or 


Y2 1s zero. Hence, the working basis changes, but we still remain at the same 


extreme point solution. Consequently, it is possible to cycle, that is, go through a 
sequence of consecutive degenerate pivots, and loop through the same set of 
working bases. Therefore, we need some cycling prevention rule as for the 
nonnegatively constrained case. For example, the following lexicographic rule may 
be used. 

Suppose that for all basic feasible solutions encountered by the algorithm, 


the rows of B™! that correspond to degenerate basic variables at their lower 


bounds are lexicographic positive, and the rows of B! that correspond to 
degenerate basic variables at their upper bounds are lexicographic negative. A 
basic feasible partition [B,N,, N»] that satisfies this property is called a strongly 


feasible basic partition. This is readily achieved to begin with by using artificial 
variables if necessary. In order to maintain strongly feasible basic partitions, the 
following rule can be adopted whenever A; < . (The reader is asked to justify 


this rule in Exercise 5.21.) Suppose that the entering variable x, is increased from 
its lower bound. If x, blocks itself (A, = uj, — ¢,) and if the resulting partition 


with the same basis remains strongly feasible, then no pivot is required. Otherwise 
(whether x, blocks itself or not), let S, be the set of basic variable indices i for 


which y;, <0 and x; is at its upper bound when x, =, + A,. Let S5 be the set 
of basic variable indices i for which y;, > 0 and x; is at its lower bound when x; = 


é, + Ay. Letting B;! denote the row of B~! corresponding to the basic variable 


x;, determine the unique index r for which By /yxs i € S; US» is lexi- 


cographically minimum, and pivot x, out of the (working) basis. Similarly, 
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suppose that the entering variable x, is being decreased from its upper bound. If 
x, blocks itself and if the resulting partition with the same basis remains strongly 
feasible, then no pivot is required. Otherwise, let S, = {i: x; is basic, yj <0, and 
x, = €; when x, = uz, —Aj;}. Let Sy = {i: x; is basic, yz >0, and x; =u; when 
xX, = uz — A;}. Then, select x, to exit from the basis where r is the unique index 
for which B;'/ Vite 1 € S, U Sy is lexicographically maximum. With this rule, 


whenever a sequence of degenerate pivots is performed, the vector ¢c ee can be 


readily verified to be lexicographically decreasing, and hence, cycling cannot 
occur. (We ask the reader to provide a proof for this in Exercise 5.21.) 

In a similar manner, we can generalize Bland’s cycling prevention rule, 
for example, to the present case. Suppose that by performing suitable 


transformations if necessary, we have the lower bound vector 2 = 0. Furthermore, 
for convenience, assume that 0 < u; < oo for each i = 1, ..., m, and suppose that the 


upper bounding constraints are written as x + s = u in the problem, where s > 0 are 
the slack variables. Note that given a basic feasible partition [B,N,, N>] for the 


system Ax = b, 0 < x < u, there is an associated basis for the nonnegatively 
constrained system Ax = b, x + s = u, x and s > 0, in which the (m + n) basic 
variables are Xp, X Ny» SB> and s Mp and the (n — m) nonbasic variables (at value 
zero) are xy, and 8y,, and vice versa. But as in Section 4.5, Bland’s Rule applied 


to the latter system would require that the variables be arranged in some order, say, 
X1y05Xp> Spo-5,, and ties for the entering and leaving candidates be broken by 


picking the first candidate from this list. The translation of this rule in the present 
context is now readily evident using the foregoing equivalence of partitions into 
sets of basic and nonbasic variables. (We leave the details to the reader in Exercise 
5.22.) 


Summary of the Simplex Method for Bounded Variables 
(Minimization Problem) 


INITIALIZATION STEP 


Find a starting basic feasible solution (use artificial variables if necessary). Let xz 


be the basic variables and let xy, and xy, be the nonbasic variables at their lower 

and upper bounds, respectively. Form the following tableau where Z = c ,B 'b + 
-l -1 , _p-l -l 

(cy, —¢gB N) én, + (cn, —CpB No )uy, and b=Bb- B Nien, - 


=] . 
B Nouy, 7 
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MAIN STEP 


l. Ifz Eps 0 for nonbasic variables at their lower bounds and z pee 


> 0 for nonbasic variables at their upper bounds, then the current 
solution is optimal. Otherwise, if one of these conditions is violated for 
the index &, then go to Step 2 if x, is at its lower bound and Step 3 if 
X, is at its upper bound. 

2. The variable x, is increased from its current value of 2; to 2; + Ag. 
The value of A, is given by Equation (5.16) where y, and y> are 
given by Equations (5.14) and (5.15). If A, = 0%, stop; the optimal 
objective value is unbounded. Otherwise, the tableau is updated, as 
described previously. Repeat Step 1. 

3. The variable x, is decreased from its current value of u, to uj, — Ag. 
The value of A, is given by Equation (5.16) where y, and v2 are 
given by Equations (5.19) and (5.20). If A, = ©, stop; the optimal 
objective value is unbounded. Otherwise, the tableau is updated as 
described previously. Repeat Step 1. 


It will be helpful to distinguish between nonbasic variables at their lower and upper 
bounds during the simplex iterations. This is done by flagging the corresponding 
columns by # and u, respectively. 


Example 5.6 
Minimize —2x, - 4x, - x3 
subjectto 2x, + Xx. + x3 S< 10 
x} ot x2 — X3 < 4 
O< x < 4 
O< oo) < 6 
l< x3 < 4, 


Introduce the slack variables x4, and x5. These are bounded below by 0 and 
bounded above by «. Initially, the basic variables are x4 and x5, and the nonbasic 
variables at their lower bound are x, = x7 =0 and x3 = 1. Note that the objective 
value is —1 and that the values of the basic variables x4 and xs are given by 10 —1 
=9 and 4+ 1=5, respectively. 
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Iteration 1 


The maximum value of z ey for lower bounded nonbasic variables is 4 corre- 
sponding to x. Therefore, x, = x» is increased. Then y> = H and A, is given 


by minimum{y,, 72, w.— 7} = minimum {7,, 7, 6}. Also, 7, and 72 are given, 
according to Equations (5.14) and (5.15), as follows. First, 
£2 9-0 5-0 
y, = minimum ap a =5 
corresponding to xg, = x5, that is, r= 2. This means that A can be increased to 


value 5 before a basic variable drops to its lower bound. Second, vy = «©, which 
means that A, can be increased indefinitely without any basic variable reaching 


its upper bound. 
Therefore, A, = minimum {5, 0, 6} = 5. The objective value is replaced 


by -1 —(2) -e)) Ay =-1-4x5=~21 and 


Es }-[s}-ve% -[s]-Lils-[o] 


The value of x2 is given by A, = 5. Moreover, x) enters and x; leaves. The 
tableau is updated by pivoting at >. 


Iteration 2 


All nonbasic variables are at their lower bounds and the maximum value of 


z; ~ c; is 5, corresponding to x3. Therefore, x3 enters, y3 = ah and A3 is 
given by 


A; = minimum {7), 72,43 — £3} = minimum {7;, 72,3}. 
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The values of y, and > are obtained from Equations (5.14) and (5.15) as follows: 
4-0 
=——=2 
‘Al 2 
corresponding to xg = xp, = x4, that is, r= 1. This means that x4 would drop to 


its lower limit if x, were increased by 2. Second, 
6-5 
ae | 
corresponding to xp = Xp, =X, that is, r= 2. This means that x. would reach its 
upper bound if x, were increased by 1. Therefore, A; = minimum {2, 1, 3} =1= 
Vy. Now, x3 =1+ Aj =2. 
The objective value is replaced by —21 — (z3 —c3) A, =-21 -5 x 1 =—26. 


X4 = 4 i _ 4 = 2 _ 2 
Es }-[5}--[3}LabL} 
Here, x3 enters the basis and x2 reaches its upper bound and leaves the basis. The 


tableau (except the RHS, which was updated separately) is updated by pivoting at 
Ing = Y23= 71. 


Iteration 3 


The maximum value of z 77S for nonbasic variables at their lower bounds is 3, 


corresponding to x. Therefore, x, is increased. Here, y, = El and A; is given 
by 
A, =minimum{y,, 72,4, — 2;} = minimum {y, 77,4}. 


The values of y, and yz are given by Equations (5.14) and (5.15) as follows. 
First, 


corresponding to xg = xg = x4, that is, r= 1. This means that if x was 


increased by 2/3, x, would drop to its lower limit and would leave the basis. 
Second, 
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a= eo 
corresponding to xg = xg, = X3, that is, r= 3. This means that if x, was 
increased by 2, x; would reach its upper limit and would leave the basis. 
Therefore, A, = minimum{2/3,2,4} = 2/3. So, x, = 2/3. The objective 


value is replaced by — 26 — (z, — ¢)A, =—26 —-3 x (2/3) =—- 


ES Fe[2}--[3}-[a]e-[a5] 


Here, x, enters the basis and x4 leaves the basis. 
The tableau is updated by pivoting at y,,; = 3, and the RHS is updated 
separately where z = —28, x, = 2/3, and x3 = 8/3. 


Iteration 4 


u £ £ 
Zz x] x2 x3 X4 “3 
z [1] : 3 : -1 


X 2/3 1/3 7 2/3 

X3 ; ~1/3 ‘ 3  -2/3 8/3 
Since z; — c; 2 0 for nonbasic variables at their upper bounds and z; — c; <0 for 
nonbasic variables at their lower bounds, then the foregoing onal gives an 


optimal solution (is it unique?). The variable values are given by (x),x2, %3,%4,X5) 
= (2/3, 6, 8/3, 0, 0) and the objective value is —28. 


5.3 FARKAS’ LEMMA VIA THE SIMPLEX METHOD 


Farkas’ Lemma is a key ingredient to establishing the Karush-Kuhn—Tucker 
optimality conditions that are both necessary and sufficient for linear programming 
problems. The result deals with two linear systems of equations and inequalities, 
exactly one of which has a solution in any instance. It turns out that the simplex 
method can be used to provide a constructive proof for this result. We state the 
lemma, give an insightful proof, and then discuss its geometric interpretation. 


Lemma 5.1 (Farkas’ Lemma) 


One and only one of the following two systems has a solution: 
System 1: Ax >0 and ex <0; 
System 2: wA =C and w>0, 


where A is a given m x n matrix and ¢ is a given n—vector. 
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Proof 


The variables in the two systems are x and w, respectively. The theorem can be 
restated as follows. If there exists an x with Ax > 0 and ex <0, then there is no w > 
0 with wA = c. Conversely, if there exists no x with Ax > 0 and ex < 0, then there 
exists a w > 0 such that wA = c. 

Suppose that System 1 has a solution x. If System 2 also has a solution w, 
then ex = wAx > 0, since w > 0 and Ax > 0. This contradicts ex < 0; therefore, 
System 2 cannot have a solution. 

Next, suppose that System 1 does not have a solution. Consider the 
Problem P: Minimize {ex : Ax > 0}. Note that zero is the optimal value for 
Problem P (why?). Putting P in standard form by writing x = x’ — x”, where x’, x” 
> 0, and denoting the surplus variable vector in the inequality constraints as s, we 
obtain the equivalent problem: 


P': Minimize {ex’— ex” : Ax'— Ax"-s=0, x’,x",s > 0}. 


Note that x’ = x” = 0, s = 0 is an optimal extreme point solution to P’. Starting 
with s as the initial set of basic variables, we can use a cycling prevention rule to 
obtain an optimal basis for P’ such that “z 777” <0 for all the variables. (Such a 


basis exists by Theorem 4.1.) Let w = “ec pB! ” be the set of simplex multipliers 
associated with this basis B, say. Since “z j7cej = wa; —c Ge < 0 for all the 
variables, we get wA — c < 0, -wA + c < 0, and —w < 0 from the columns of the 
variables x’, x", and s, respectively. Hence, we have constructively obtained a w 


> 0 such that wA = c. The proof is complete. 
Alternative Forms of Farkas’ Lemma 


Several variants of Lemma 5.1 exist that deal with other pairs of systems of 
equations and inequalities, exactly one of which has a solution in any instance. 
These results are collectively known as Theorems of the Alternative. We state one 
such variant as Corollary 5.1 for the sake of illustration and present another form 
of this result in Exercise 5.34. 


Corollary 5.1 (Alternative Form of Farkas’ Lemma) 
One and only one of the following two systems has a solution: 


System 1: Ay <0, y <0, and cy > 0; 
System 2: wA <¢ and w>0. 


This form can be easily deduced from the format in Lemma 5.1 by changing 
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System 2 into equality form. Since wA < ¢ and w > 0 is equivalent to (w, w(t) =e 


and (w, v) > (0, 0), then from Lemma 5.1, System 1 must read (tk > (0) and ex < 
0, that is, Ax > 0, x > 0, and ex < 0. Putting y = —x gives the desired result. 


Geometric Interpretation of Farkas’ Lemma 


Denote the ith row of A by a’, i= 1, ..., m, and consider System 2. Here, wA =c¢ 
and w > 0 simply means that ¢= >", wal, w; = 0 for i= 1, ..., m. In other words, 


System 2 has a solution if and only if ¢ belongs to the cone generated by the rows 


of A, namely the vectors al ..., a”. Hence, Farkas’ Lemma asserts that either ¢ 


belongs to the cone generated by the rows A or it does not. In the former case, 
System 1 has no solution, and in the latter case, it does. For convenience in 
illustration, let us equivalently restate System 1 as requirimg to find a solution y 
satisfying Ay < 0 and cy > 0 (put y = —x). Geometrically, such a vector y should 


make an angle > 90° with each row of A since aly < 0 for i = 1, ..., m, and it 


Cone A gp Open half-space 


Figure 5.2. System 1 has a solution. 


Open half-space 


Cone 


Figure 5.3. System 2 has a solution. 
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should make an angle < 90° with the vector ¢ since cy > 0. Therefore, System | has 
a solution if and only if the intersection of the cone {y: Ay < 0} and the open half— 
space {y: cy > 0} is not empty. Figure 5.2 shows a case where System 1 has a 
solution (with any y in the shaded region yielding a solution x = —y to System 1). 
Figure 5.3 shows a case where System 2 has a solution. Accordingly, System 2 has 
no solution in Figure 5.2, and System 1 has no solution in Figure 5.3. 


5.4 THE KARUSH-KUHN-TUCKER OPTIMALITY CONDITIONS 


The Karush-Kuhn-Tucker (KKT) conditions form the backbone of nonlinear pro- 
gramming in general. These conditions are necessary for optimality for differentiable 
nonlinear programming problems that satisfy certain regularity conditions (known as 
constraint qualifications) and suffice to indicate optimality for certain differentiable 
nonlinear programming problems that satisfy various generalized convexity 
properties (see the Notes and References section). For linear programming problems, 
these conditions are both necessary and sufficient, as we shall be seeing, and hence 
form an important characterization of optimality. These conditions will be widely 
used in the remainder of this book. Later, we present a simple, geometrically 
motivated derivation of this result for linear programming problems. (In fact, this 
derivation can be readily generalized to obtain necessary optimality conditions for 
differentiable nonlinear programming problems under suitable constraint 
qualifications by considering appropriate first-order linearizations at an optimal 
solution.) 


The Karush-Kuhn-Tucker Conditions for Inequality 
Constrained Problems 


Consider the following linear programming problem: 
Minimize ex 
subjectto Ax>b 
x>0, 


where ¢ is an #—-vector, b is an m—vector, and A is an m x n matrix. Let X be any 
feasible solution to this problem. Denote by Gx > g the set of inequalities from Ax 
> b and x > 0 that are binding at x. Observe that if Xis an optimal solution, then 
there cannot be any improving feasible direction d at x. Mathematically, there 
cannot be a direction d such that ed < 0 and Gd > 0. Otherwise, by moving along 
such a direction from x, the objective function value would fall, since ed < 0, and 
we would remain feasible to the constraints Gx > g, since G(x + Ad) = Gx + AGd 
= g +AGd > g, for any 1 > 0. Moreover, for some step length 2 > 0, we would also 
remain feasible to the constraints that are nonbinding at x. Since the system cd < 0 
and Gd > 0 cannot have a solution, by using Farkas’ Lemma, we deduce that there 
exists a u > 0 such that uG = c. Let us now rewrite this statement in terms of the 


constraints Ax > b and x > 0. Toward this end, let a’ be the ith row of A,i=1,..., 
m, and let e; be a vector of zeros except for a | in the jth position. Define 


T= {i:a'X¥ =} 
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as the set of binding constraints at x from the set Ax > b, and define 

J ={j:x; =0} 
as the set of binding constraints at x from the set x > 0. Hence, the rows of G are 
comprised of a’, i J, and e;, Jj € J. Accordingly, letting u = (w; fori € J, v; 


for j € J) >0, we can rewrite the conditions uG = c, u> 0 as 


Dr wa'+ D ve; =e (5.21) 
iel jet 
w; 2Ofori e J, and vi >Ofor j € J. (5.22) 


Equations (5.21) and (5.22), along with the feasibility of k are the KKT conditions 
for the foregoing linear programming problem. So far, we have shown that these 
conditions must necessarily hold at an optimal solution x to this problem. 
Geometrically, Equations (5.21) and (5.22) assert that the objective gradient ¢ can 


be represented as a nonnegative linear combination of the gradients of the binding 


constraints at an optimal solution, where a! is the gradient of the constraint alx > 
b and e; is the gradient of the constraint x; 2 0. In other words, Equations (5.21) 


and (5.22) say that the objective gradient ¢ must lie in the cone generated by the 
gradients of the binding constraints. 

Conversely, suppose that Equations (5.21) and (5.22) hold at some 
feasible point x with / and J as defined. Consider any other feasible solution xX to 
the linear program. Then, multiplying both sides of Equation (5.21) by (x — x) 


and using the fact that a/X = b; fori € J and ex =0 for j € J, we obtain 


ek eX = Y w,(a'k-4))+ ¥ ve X20 
iel jeJ 


since aX > b; fori € I, e;x20 for 7 € J, and Equation (5.22) holds true. Hence, 


ex 2>cx for any feasible solution x (in fact, feasible simply to the inequalities 
binding at x), and so, x is an optimal solution. 

Consequently, we have shown that a feasible solution x is optimal to the 
foregoing linear program if and only if Equations (5.21) and (5.22) hold true, that 
is, if and only if the objective gradient ¢ lies in the cone generated by the gradients 
of the binding constraints at x. Hence, the KKT conditions given by Equations 
(5.21) and (5.22) along with the feasibility of x are both necessary and sufficient 
for the optimality of x. These conditions are usually written in the following 
convenient form. Define the vectors w = (wy, ..., Wy,) 20 and v =(y, ..., ¥,) 20. 


Note that w; and v; must be zero corresponding to the nonbinding constraints. 


Under this restriction, Equation (5.21) can be written as wA + vI =, or simply, 
wA + v =c. Hence, the KKT conditions can be equivalently stated as requiring a 
solution (x, w, v) to the following system: 


Ax >b, x>0 (5.23) 
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wA+v=c, w2>0,v20 (5.24) 
w(Ax — b) =0, vx = 0. (5.25) 


The first condition (5.23) merely states that the candidate point must be feasible; 
that is, it must satisfy the constraints of the problem. This is usually referred to as 
primal feasibility. The second condition (5.24) is usually referred to as dual 
feasibility, since it corresponds to feasibility of a problem closely related to the 
original problem. This problem is called the dual problem and will be discussed in 
detail in Chapter 6. Here, w and v are called the Lagrangian multipliers (or 
Lagrange multipliers, or dual variables) corresponding to the constraints Ax > b 
and x > 90, respectively. Finally, the third condition (5.25) is usually referred to as 
complementary slackness. Since w > 0 and Ax > b, then w(Ax — b) = 0 if and only 
if either w; is 0 or else the ith slack variable is 0, that is, the ith constraint in Ax > 


b is binding. Similarly, vx = 0 if and only if either x; is 0 or else v; is 0. Hence, 


Equations (5.24) and (5.25) are equivalent to Equations (5.21) and (5.22). 
The following example illustrates the Karush-Kuhn—Tucker conditions. 


Example 5.7 
Minimize -—x,; — 3x 
subjectto x, — 2x, = -4 
-X) -— xX 2 4 
Xi, X, 2 0. 


Equations (5.23), (5.24), and (5.25) represent a useful tool in verifying whether a 
certain point is optimal. To illustrate, suppose we were told that the optimal 
solution of the foregoing problem is the point (0, 0). We see geometrically in 
Figure 5.4 or by using the simplex method that (0, 0) is not an optimal solution. 
First, inequality (5.23) holds since (0, 0) is indeed a feasible solution. Since none 
of the first two constraints is binding (that is, x, — 2x. > —4 and — x, — xp > —4), 


then w, = w = 0, in order to satisfy Equation (5.25). Since w = 0, then from 


: | 


Figure 5.4. Verification of the KKT conditions. 
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Equation (5.24) we must have ¢ = v, that is, v = (-1, -3). However, this violates 
the nonnegativity of v. Therefore, (0, 0) could not be an optimal solution for this 
problem. 


Now, suppose that we were told that the optimal point is (4/3, 8/3). In 
order to check whether this is a true statement, we can use the KKT conditions. 
Since x;, x2 > 0, then v, = v, = 0 in order to satisfy complementary slackness. 


From Equation (5.24), w must satisfy the system ¢ — wA = 0, that is, 


ele ala 
2 TW = 3 
and hence w, = 2/3 and w, = 5/3. Note that w > 0 and Ax = b, and hence, w(Ax ~ 


b) = 0. Therefore, conditions (5.23), (5.24), and (5.25) hold true, and (4/3,8/3) is 


indeed an optimal solution. 
The following example illustrates the geometric interpretation of the KKT 
conditions. 


Example 5.8 
Minimize -x,; — 3x 
subjectto x, -— 2x, 2 -4 
“Xp x4 => —-4 
xX; x4 > 0. 
The gradients of the objective function and the constraints are given below: 
e = (-1, -3) 
al = (1, -2) 
a’ = (-1, -1) 
a (1, 0) 
& = (0, 1). 


Let us consider the four extreme points of Figure 5.5. 


1. The extreme point x = (0, 0)’: The binding constraints are the 
nonnegativity constraints. We see from Figure 5.5 that ¢ does not 
belong to the cone of the gradients e; and e, of the binding constraints. 
Therefore, (0,0)' is not optimal. 


2. The extreme point x = (0,2)': The binding constraints are x, — 2x > 
—4 and x, > 0. Here, ¢ does not belong to the cone generated by the 


gradients a! and e, of the binding constraints. So, (0,2)' is not optimal. 
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4/3 
8/3 


Figure 5.5. Geometry of the KKT conditions. 


3. The extreme point x = (4/3,8/3)': The binding constraints are x — 2X, 
> —4 and —x, — x) => —4. Here, ¢ belongs to the cone of the gradients al 


and a” of the binding constraints. Therefore, (4/3,8/3)' is an optimal 
point. 

4. The extreme point x = (4,0): The binding constraints are — x, — x, > 
-4 and xy > 0. Here, ¢ does not belong to the cone generated by the 


gradients a’ and e, of the binding constraints, and hence, (4,0)' is not 
optimal. 


The Karush-Kuhn—Tucker Conditions for Equality Constraints 
Consider the following linear programming problem in equality form: 
Minimize cx 
subjectto Ax=b 


x>0. 


By changing the equality into two inequalities of the form Ax > b and —Ax > -b, 
the KKT conditions developed earlier would simplify to 


Ax=b, x>0 (5.26) 
wA+v=c, Ww unrestricted, v>0 (5.27) 
vx = 0. (5.28) 


The main difference between these conditions and the conditions for the inequality 
problem is that the Lagrangian multiplier vector (or dual vector) w corresponding 
to the constraint Ax = b is unrestricted in sign. 
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Optimality at a Basic Feasible Solution 
Consider the following problem: 

Minimize ex 

subjectto Ax=b 

x>0. 

Assume that rank (A) = m. Let us reinvestigate how the simplex method recognizes 
an optimal basic feasible solution. Suppose that we have a basic feasible solution x 
with basis B and let us examine conditions (5.26), (5.27), and (5.28). Obviously, 


(5.26) holds true. The condition e — wA — v = 0 can be rewritten as follows, where 
Vv is decomposed into vz and vj: 


(Cp, Cy) — W(B, N)— (vz, Va) = (0, 9). (5.29) 


To satisfy the complementary slackness condition vx = 0, since x,y = 0, it suffices 
to have vz = 0, so that vx = vpxp + VyXy = 0. With vp = 0, Equation (5.29) 
reduces to the following two equations: 

cz ~ WB=0 

Cy —WN- vy=90. 
From the first equation we get w =c ,B, and from the second equation we get 
Vy = Cy —WN= Cy — cpB"!N. 


To summarize, given a basic feasible solution, condition (5.26) automatically holds 
true. Equation (5.28) is satisfied by letting vz = 90, and the condition WA + v= c is 
satisfied by letting w = cpB! and Vy = cy — cpB"!N. The only possible 
source of violation of the Karush—Kuhn-Tucker conditions is that v,, might not be 
nonnegative. Note, however, that v,, consists of the (c ers ‘;)—values for the 


nonbasic variables. Therefore, the nonnegativity of v, in Equation (5.27) is 


violated if c pe 0 for some nonbasic variable. Of course, if c yp 7aye 0 for 


each nonbasic variable, then v, > 0 and all the Karush-Kuhn—Tucker conditions 
are met. These are precisely the simplex termination criteria. 


Reinterpretation of the Simplex Method 


From the previous discussion, the simplex method can be interpreted as a 
systematic procedure for approaching an optimal extreme point satisfying the 
Karush—Kuhn-Tucker conditions. At each iteration, feasibility (called primal 
feasibility) is satisfied, and hence, condition (5.26) always holds true. Also, 
complementary slackness is always satisfied, since either a variable x; is nonbasic 


and has value zero, or else 1 aa ee 7 te 0, and hence Vee p= 0 for all /, i.e., 
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vx = 0. Therefore, condition (5.28) is always satisfied during the simplex method. 
Condition (5.27) (called dual feasibility; more on this in Chapter 6) is partially 
violated during the iterations of the simplex method. Condition (5.27) has two 
portions, namely, wA + v = ¢ and v > 0. The first portion always holds true by 


letting w = cpB ! and v= (vz, Vy) =(0, cy —¢pB 'N). However, the second 


portion, namely nonnegativity of c, —¢ 3B !N, which is part of dual feasibility, is 
violated, until of course, an optimal basis is reached. To summarize, the simplex 
method always satisfies primal feasibility and complementary slackness. Dual 
feasibility is violated, and the violation is used to try and improve the objective 
function value by increasing the nonbasic variable having the most negative 
ae aT for example, until dual feasibility is also satisfied. 


Finding the Lagrangian Multipliers from the Simplex Tableau 


We already know that vz = 0 and vy =ey —¢ 3B IN . Therefore, the Lagrangian 


multiplier v; corresponding to the constraint x j 20 can be easily obtained from 


row 0 of the simplex tableau. More precisely, v; is the negative of the z; — c; 


entry in row 0 under the x, column. 


Now, we turn to the Lagrangian vector w = ¢ BB! corresponding to the 


constraints Ax = b. The method for obtaining w from the simplex tableau was 
discussed in Chapter 3. We shall elaborate on this further. Recall that row 0 of the 
simplex method consists of z; — c; for j = 1,...,.n, which is given by the vector 


c ,B'A — c. If the matrix A has an identity matrix as a portion of its columns, 


then in row 0 under these columns, we have cpB I -— ¢ =w— ¢, where ¢ is the 


part of the cost vector ¢ corresponding to the identity columns in the original 
problem. By simply adding the vector ¢ tow — € in row 0, we get w. 


EXERCISES 


[5.1] Solve the following linear program by the revised simplex method: 


Minimize x, + 4x5 - 7x3 + X4 + 5x5 
subjectto x, — (3/4)x> + 2x3 - (1/4)x4 

— (1/4)x. + 3x3 - (3/4) xq + Xs 

X; x2, X3; X45 Xs 


5 
0. 


IV Ib ul 


[5.2] Solve the following linear program by the revised simplex method: 


Maximize - 2x% + % 

subjectto x, — 2x, + x3 2 -4 
XN +t XxX + X% < 7 
2x, — XX. - X%3 < 5 
X1> X92; X3 2 0 
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[5.3] Solve the following problem by the revised simplex method using the 
product form of the inverse: 


Maximize 3x, + 2x7 + x3 + x4 

subject to 8x, + 3x, + 4% + 4% <7 
2X, + X + 4% + Sxq < 3 
xX + 4x. + 3x3 + 2x4 < 8 
xX, Xp, x3, x4 2 0. 


[5.4] Repeat Exercise 5.3 using an LU-—factorization of the basis. 
[5.5] Solve Exercise 5.1 using the product form of the inverse. 
[5.6] Verify the entries in Table 5.1. 


[5.7] An automobile manufacturer has contracted to export 400 cars of model A 
and 500 cars of model B overseas. The model A car occupies a volume of 12 cubic 
meters, and the model B car occupies a volume of 16 cubic meters. Three ships for 
transporting the automobiles are available. These arrive at the port of destination at 
the beginning of January, the middle of February, and the end of March, 
respectively. The first ship only transports model A cars at $450 per automobile. 
The second and third ships transport both types at a cost of $35 and $40 per cubic 
meter, respectively. The first ship can only accommodate 200 cars, and the second 
and third ships have available volumes of 4500 and 6000 cubic meters, 
respectively. If the manufacturer has contracted to deliver at least 250 and 200 cars 
of models A and B by the middle of February and the remainder by the end of 
March, what is the shipping pattern that minimizes the total cost? Use the revised 
simplex method. 

[5.8] Apply the revised simplex method to the following problem, both with and 
without the lexicographic cycling prevention rule. (This is the example used in 
Chapter 4 to illustrate cycling.) 


Minimize — (3/4)x4 + 20x, - (1/2)x5 + 6x, 
subject to x + (1/4)xq - 8x5 - Xe + 9x7 = 0 
XQ + (1/2)xq4 - 12x5 - (1/2)x5 + 3x, = 0 
x3 + X6 = 1 
x, X25 3, X4, X5, X6; xX, 2 0. 


[5.9] In the revised simplex method with the product form of the inverse, the 
number of elementary matrices increases by | at each iteration. If the number of 
elementary matrices becomes excessive, it would be necessary to reinvert B. Let B 


be a basis. Show how B can be reinverted such that B™! is represented as the 
product of m elementary matrices. Illustrate by an example. 


[5.10] Determine the number of multiplications and additions needed per iteration 
of the revised simplex method using the product form of the inverse. How can we 
take advantage of sparsity of nonzero elements in the matrix A? Give a detailed 
comparison between the simplex method and the revised simplex method using the 
product form of the inverse. Repeat using an LU-factorization of the basis. 
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[5.11] How would you use the revised simplex method to solve a bounded 
variables linear programming problem? Consider the following bounded variables 
linear program: 


Maximize z = % — 2x 
subject to —% - 2% S 1 
2x, = x2 <1 
2< x < il 
x2 < 2. 


a. Draw the feasible region in the (x;, x2) space and identify the optimal 
solution. Is this degenerate? Why or why not? 

b. For the extreme point (x, x2) =(1, 1), identify the working basis B 
and set up the revised simplex tableau by determining Bic 2B, the 
objective function value, and the basic variable values. Continue 
solving the problem from this basic feasible solution using the bounded 
variables revised simplex algorithm. 

c. Denote the right—hand-sides of the first two constraints by }, and dy. 
What conditions must 5, and 5, satisfy for the optimal partition to 
remain optimal? 

[5.12] Use the simplex method for bounded variables to solve the following 
problem: 


Minimize x + 2x2 + 3x3 - % 
subject to 2x; -— Xp + XO 2X4 < 6 
~“xXy + 2x2 — X3 + X% < 8 
2x + Xp -— X3 > 2 
O< xy < 3 
l< oo) < 4 
0< x3 < 8 
2< X4 <5 


[5.13] Use the simplex method for bounded variables to solve the following 
problem: 


Maximize 2x, + x2 + 3x3 
subject to 3x) + X2 + x3 < 12 
xy + x2 < 4 
x2 + 2X3 < 8 
Oo< xX < 3 
Oo< x2 < 5 
O0o< X3 < 4. 
[5.14] Solve the following problem by the simplex method for bounded variables: 
Maximize 2x, + 3x2 -— 2x3 
subjectto x, + 3x7 + x3 < 8 
2x] + x2 - X3 > 3 
xy < 4 
—2 < x2 < 3 
X3 = 2 
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[5.15} A manufacturing firm would like to plan its production/inventory policy for 
the months of August, September, October, and November. The product under 
consideration is seasonal, and its demand over the particular months is estimated to 
be 500, 600, 800, and 1200 units, respectively. Currently, the monthly production 
capacity is 600 units with a unit cost of $30. Management has decided to install a 
new production system with a monthly capacity of 1200 units at a unit cost of $35. 
However, the new system cannot be installed until the middle of November. 
Assume that the starting inventory is 250 units and that at most 400 units can be 
stored during any given month. If the holding inventory cost per month per item is 
$5, find the production schedule that minimizes the total production and inventory 
cost using the bounded simplex method. Assume that demand must be satisfied 
and that 100 units are required in inventory at the end of November. 


[5.16] a. Solve the following (knapsack) problem. 
Maximize 2x, + 3x + 8x3; + X4 + 5X5 


subject to 3x, + Sx, + Ilxy + 2x4 + 7x5 
> *2> %3, X4, x5 


VIA 
—_ 
fc) 


b. Give a generalized closed-form solution for the following problem in 
terms of the ratios cjlaj; j=l... 

Maximize cx; t+ CyX, 

subject to ajxy +++ ayXy 

Xy, sey Xp 


IV IA 


0, 
where c; and a; are positive scalars for each j. 
c. What is the form of the optimal solution if c; and a; are allowed to 


be any scalars in Part (b)? 
d. Extend Parts (b) and (c) to include upper bounds on the variables. 


[5.17] Consider the problem: Minimize ex subject to Ax = b, 2< x < u. Show in 
detail that the collection of extreme points and the collection of basic feasible 
solutions as defined in Section 5.2 are nonempty and equal, given that the problem 
is feasible and that 2 is finite. 


[5.18] For the bounded variables linear program, what is the relationship between 
basic feasible partitions [B, N,;, N2] and extreme points? What is the relationship 
between adjacent basic feasible partitions and adjacent extreme points, both for the 
nondegenerate and the degenerate cases? 

[5.19] Consider the problem: Minimize ex subject to Ax = b, 0 < x < u. Show that 
the basic feasible solutions defined in Section 5.2, and the basic feasible solutions 
that would be obtained if the constraint x < u is transformed into x + x, = u and 


x, 20 are equivalent. 


[5.20] Compare the simplex method of Chapter 3 with the lower—upper bounded 
simplex method. Determine the number of operations (additions, multiplications, 
and so on) when each of the two methods is applied to the same lower—upper 
bounded linear program. 
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[5.21] Show in detail that the simplex method for bounded variables discussed in 
Section 5.2 converges in a finite number of steps in the absence of degeneracy. 
Discuss in detail the problem of degeneracy and prove that the lexicographic rule 
given in Section 5.2 maintains strongly feasible basic partitions and prevents 
cycling. 

[5.22] Using the characterization in Exercise 5.19 for the problem: Minimize cx 
subject to Ax = b, 0 <x <u, and the discussion on Bland’s cycling prevention rule 
in Section 4.5, devise a version of Bland’s Rule for this bounded variables linear 
program. 


[5.23] Illustrate the lexicographic as well as Bland’s cycling prevention rules on 
the following bounded variables linear program: 


Maximize z = 2x, + Xy 

subject to -2x, + Xx < 0 
2x, + 3x, < 6 
x - xX» Ss | 


O<x,sl Ox, <3/2. 


[5.24] Solve the following problem by the simplex method for bounded variables: 


Maximize 6x; + 4x) + 2x3 
subject to 4x, — 3x, + x3 < 6 
3x, + 2x, + 4x3 < 8 
O< x) < 3 
O< X92 < 2 
O< X3 < ©, 


[5.25] Solve the following problem by the simplex method for bounded variables: 


Minimize 2x, + 6x. - x3 -— 4x4 + Xs 
subject to 2x, + X, + 4x3; + Xx4 + x5 = 10 
2x, + 8x, — 3x, + X% = 7 
O< x} < 3 
l< Xo < 4 
O< X3 < 8 
l< X4 < 2 
O< Xs < 4. 
[5.26] Consider the following problem: 
Minimize x, + 3x, + 4x3 
subjectto -—x,; — 2x, — x3 < —-12 
x aa x2 + 2X3 < -6 
3x, = 3x an 4x3 < -24 
X1> X2> X3 2 0. 


Let xg be an artificial variable with an activity vector b<b= (-12, -6, ay 
Introducing the restrictions 0 < xg <1 and letting x, = 1 would lead to a starting 


basic feasible solution of the new system. Use the bounded simplex method to find 
a basic feasible solution of the original system. 


[5.27] Solve the following problem by the simplex method for bounded variables: 
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Minimize 2x, + x) 


subjectto x + x» 2 4 
xX) — 2x, < 0 

x1 = 2 

x2 > 1. 


[5.28] Solve the following problem by the simplex method for bounded variables. 
Begin with x, at its lower bound and x, at its upper bound. 


Maximize 3x, + 2x 


subjectto x + 2x, < 8 
xy + XxX < 5 
O< x <3 
O< x2 < 4. 


[5.29] Solve Exercise 1.12. 


[5.30] A farmer who raises chickens would like to determine the amounts of the 
available ingredients that will meet certain nutritional requirements. The available 
ingredients and their costs per serving, the units of nutrients per serving of the 
ingredients, and the units of daily nutrient requirements are summarized below: 


INGREDIENT MINIMUM DAILY 
NUTRIENT CORN LIME ALFALFA REQUIREMENT 
Protein 8 4 4 10 
Carbohydrates 4 2 4 6 
Vitamins 2 3 4 5 
$cost/serving 0.15 0.08 0.05 


Find an optimal mix using the revised simplex method with the product form of the 
inverse. Use only one artificial variable. 
[5.31] Show that the following two problems are equivalent: 
Py : Minimize cx Py : Minimize ex 
subject to by < Ax < bo subject to Ax + § 
x 20. 
0 <s <b —- dy. 


Use the simplex method for bounded variables to solve the following problem after 
reformulating it as above: 


Minimize 3x, — 4x2 
subjectto 3 < x4 + Xx < 4 
-15 < 3x, — 5x. < 2 
xX; x2 > 0 


[5.32] A government has allocated $20 billion of its budget for military purposes. 
Sixty percent of the military budget will be used to purchase tanks, planes, and 
missile batteries. These can be acquired at a unit cost of $600,000, $2 million, and 
$800,000, respectively. It is decided that at least 200 tanks and 200 planes must be 
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acquired. Because of the shortage of experienced pilots, it is also decided not to 
purchase more than 300 planes. For strategic purposes the ratio of the missile 
batteries to the planes purchased must fall in the range from 1/4 to 1/2. The 


objective is to maximize the overall utility of these weapons where the individual 
utilities are given as 1, 3, and 2, respectively. Find an optimal solution. 


[5.33] Let O= {1,...n}, RB <Q with P OP; =© for i,j = 1,....r and i #/j, and 


r 
U P; =Q. Develop an efficient method to solve the following problem 
i=l 
where Cre 0 for each j: 


Maximize > c;x; 


gor! 
subjectto by < Lx; < bo 
jeQ 
b < x. xy < by, Vie=1,..,7 
Jef 


O<x;su;, VjEQd. 


Apply the method to the following problem: 


Maximize 10x, + 6x, + 3x3; + Sx4 + 8x5 
subject to 30 <x + Xp + Xe + Xp + XS < 100 
26x + x4 < 40 
25 < x3 + XxX, + Xe < 80 
O< x, X45 xy < 30 
0 < Xp, X3 < 20. 


[5.34] Suppose that the following system has no solution: 


Ax =0,x>0, and ex >0. 


Devise another system that must have a solution. (Hint: Use Farkas’ Lemma.) 


[3] 
[5.35] Let A= 0 4 and ¢ = (~1, 5). Which of the following two systems has 
-1 4 


a solution? 
System 1: Ax <0, cx > 0; 
System 2: wA=c w>0. 
Illustrate geometrically. 


[5.36] Consider the problem: Minimize ex subject to Ax = b, x > 0. Let B be a 
basis. After adding the redundant constraints x, — x, = 0, the following 


equations represent all the variables in terms of the independent variables xj: 
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The simplex method proceeds by choosing the most positive z; —c;, Say, 24 —Cy. 


poe 
Then x, enters the basis and the usual minimum ratio test indicates that xg, 
leaves the basis. The foregoing array can be updated by column pivoting at y,, as 


follows: 
1. Divide the Ath column (pivot column) by —y,,. 


2. Multiply the Ath column (pivot column) by yy and add to the jth 
column. 

3. Multiply the kth column by b, and add to the right-hand-side. 
Remove the variable x, from the list of nonbasic variables and add xp. 


instead in its place. Note that no row designations are changed. 
This method of displaying the tableau and updating it is usually called the 
column simplex method. Show that pivoting gives the representation of all 
the variables in terms of the new nonbasic variables. In particular, show that 


pivoting updates the tableau such that the new cB 'N — ty, BN, 
B |b, and cpB 'b are immediately available. 


[5.37] Referring to Problem 5.36, solve the following problem using the column 
simplex method: 


Maximize x, + 2x, + 3x; 

subjectto 3x, + 2x, + 4x3 < 6 
~Xy + 3x2 + 4x, < 8 
2x + Xp -— xX < 2 
xX}, x2, x3 > 0. 


[5.38] Referring to Problem 5.36, is it possible to extract the inverse basis from a 
typical column simplex tableau? If so, how can this be done? 

[5.39] Consider the problem: Minimize ex subject to Ax = b, x > 0. Suppose that x 
is an optimal solution. Further suppose that x; > 0 forj=1,...,.p, and x; = 0 forj = 


d, > 0, and ed < 0 has no 


n= 


pt z,..., 2. Show that the system Ad = 0, A pst sn, 


solution d in R”. Hence, use Farkas’ Lemma to derive the KKT conditions for this 


problem. Also, directly show that any solution to these KKT conditions is optimal 
for this problem. 


[5.40] Consider the following problem: 
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Maximize x, + 2x 


subject to —2x, + Xx < 2 
x2 < 4 

xy + % < 5 

X> xX, 2 0. 


Starting at the origin, solve the problem by the simplex method, and exhibit that 
the optimal solution satisfies the KKT conditions. At each iteration, point out the 
source of violation of the optimality conditions. Interpret the KKT conditions 
geometrically at each extreme point encountered. 


[5.41] Write the KKT optimality conditions for each of the following problems: 


a. Maximize cx 
subject to Ax < b 
x 2 


b. Maximize ex 
subject to Ax = b 
x 2 


c. Minimize ex 
subject to Ax < b 
x 2 


d. Minimize cx 


subject to A,x = b, 
Ax 2 by 
x 2 0. 

e. Minimize ex 
subjectto Ax = b 
fsx <u 


[5.42] Consider the problem: Maximize cx subject to Ax < b, x > 0. Introducing 
the slack vector x,, we get the equivalent problem: Maximize cx subject to Ax + 


x, =b,x2>0, x, 20. Write the Karush-Kuhn—Tucker optimality conditions for 


both problems. Show equivalence of these optimality conditions, and show that the 
Lagrangian multipliers corresponding to the nonnegativity constraints x, > 0 are 


equal to the Lagrangian multipliers of each of the constraint sets Ax + x, = b and 
Ax <b in the two respective problems. 


[5.43] Consider the problem: Minimize cx subject to Ax > b, x > 0. Let x” be an 


optimal solution. Suppose that A is decomposed into a and b is decomposed 


into B such that A,x” = b, and A>x” > by. Show that x” is also an optimal 
2 


solution to the problem: Minimize cx subject to A,;x = b,, x >= 0, and to the 
problem: Minimize cx subject to Ajx = b,;, x20. 
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[5.44] Consider the linear programming problem: Minimize cx subject to Ax = b, 
x > 0. It is well known that a feasible point x is an optimal solution if and only if 
there exist vectors w and v such that 


c-wA-v = 0 
v2o0 
vx = 0. 


Is it possible that x is optimal if v; > 0 for all i #/, for some /, whereas v ; <0 and 
the corresponding x; = 0? In other words, is it possible to have an optimal 


solution with one of the Lagrangian multipliers (shadow prices) of the 
nonnegativity constraints being negative? Explain why or why not. Illustrate by a 
numerical example. (Hint: Construct a linear program with an optimal degenerate 
basic feasible solution. Try different basis representations. See Exercise 3.43b.) 


[5.45] Consider the following problem: 


Maximize 2x, a 2x + X3 

subjectto x + XxX, + x3 < 8 
3x) + X> < 4 
—X% + 2x, - % < 4 
x, x9, x3 2 0. 


Solve the problem by the simplex method, illustrating at each iteration any source 
of violation of the KKT conditions. 


[5.46] Consider the following problem: 


Maximize 2x, + 3x 


subjectto x + XxX < -8 
-2x, + 3x < 0 
x xX, 2 0. 


a. Solve the problem geometrically. Starting at the origin, at each iteration 
identify the variable that enters and the variable that leaves the basis. 
b. Write the KKT conditions and show that they hold at optimality. 


[5.47] Convert the KKT conditions for the linear programming problem: 
Minimize {cx : Ax > b, x > 0} to equivalently requiring a solution to a system of 
linear inequalities. Is there a computational advantage in such a transformation? 
[5.48] A manufacturer produces two items with unit profits $12.00 and $15.00. 
Each unit of item 1 uses 4 man-hours and 3 machine-hours. Each unit of item 2 
uses 5 man-hours and 4 machine-hours. If the total man—hours and machine-hours 
available are 300 and 450, respectively, find an optimal solution and verify 
optimality by the KKT conditions. Interpret the optimality conditions geometrically. 
Give an economic interpretation of the KKT conditions at the optimal point. (Hint: 
Recall the economic interpretation of w, and w.) 


[5.49] Consider the problem: Minimize cx subject to Ax = b, x > 0. Given a 
feasible point x, formulate a problem that will find a feasible direction that best 
improves the objective function for a unit step length, if one exists. How does this 
problem relate to the original problem? 
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[5.50] Consider the linear programming problem: Minimize cx subject to Ax = b, 
x > 0. Let x be a basic feasible solution with basis B. The simplex method (with 
Dantzig’s Rule) proceeds by increasing a nonbasic variable having the most 


positive z 


a. 


-—C>;, 


J 


Devise a procedure in which all nonbasic variables with positive 
(z ey )-values are increased. How are the basic variables modified? 


By how much would you increase the nonbasic variables? Interpret 
increasing several variables simultaneously. 

At some iterations we may have more than m positive variables. How 
would you represent the corresponding point in a tableau format? (Hint: 
Nonbasic variables may be either positive or zero.) 

At some iteration you may have a positive nonbasic variable that has a 


negative z;—c,;. What happens if you decrease x pus 


Use the ideas of Parts (a), (b), and (c) to construct a complete algorithm 
for solving linear programs where several nonbasic variables are 
simultaneously modified. How do you recognize optimality? What are 
the advantages and disadvantages of your procedure? 

Illustrate your procedure by solving the following problem: 


Minimize -3x, -— 2x. - % 

subjectto 2x, + X2 + 3x, < 12 
xy ot 2X9 < 6 
2x + x3 < 8 
X15 x9, X3 > 0. 


Consider the following alternative procedure for modifying the 
nonbasic variables. For each nonbasic variable x ,, let 


Jd > 
Snes Zj-¢; ifx; >0 
J) maximum {0,z;-c,;} ifx,; =0. 
Modify the nonbasic variables according to the d ;—values and the basic 
variables according to the relationship 
xp =b-A) yd; 
jet 


where J is the index set of the nonbasic variables, the vector b represents 
the current values of the basic variables, and 4 > 0 is to be determined. 
Interpret this method and compare it with the method in Part (d). Solve 
the problem in Part (e) by this procedure. 


[5.51] The accompanying diagram depicts the region given by a,x, + a)x2 <b 


and x;, xX, > 0. Let (x, x2) be the shown point. Indicate on the diagram the 


value of the slack variable. How can you generalize the result to n variables? 
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2 


a , 


[5.52] The following is an idea of a graphical example of the simplex method at 
work: 


x2 


Optimal solution 


x 


a. Give the starting basis and each succeeding basis until the optimal point 
is reached. Specify the entering and leaving variables. 

b. Ifthe optimal point is unique, could the simplex method have gone in 
the direction it did assuming that the entering variable is that with the 
most positive z; —c;? Explain. 


[5.53] Consider the problem: Minimize ex subject to Ax = b, x > 0. Let x” be the 
unique optimal extreme point. Show that the second best extreme point must be 


adjacent to x". What happens if the uniqueness assumption is relaxed? 


[5.54] Suppose that we are given an extreme point x of a polyhedral set X. Con- 
sider the following alternative definition to that given in Section 2.6. An extreme 
point y # x is called adjacent to x if there exists a hyperplane that supports X and 
its intersection with X is the line segment joining x and y. In the accompanying 
diagram, obviously x2 and x5 are extreme points of X that are adjacent to x,, 


whereas x3 and x4 are not adjacent to x,. Now, let X consist of all points 


satisfying Ax = b and x > 0, where A is an m x n matrix with rank m. Further 
suppose that X is bounded. Let x be a nondegenerate basic feasible solution 
(extreme point). Characterize the collection of adjacent extreme points. What is 
their number? What happens if the nondegeneracy assumption is relaxed? In each 
case justify your answer. Also, formally establish the equivalence of this definition 
with that in Section 2.6. 
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X4 x3 


[5.55] Referring to Exercise 5.54, show that the simplex method moves from an 
extreme point to an adjacent extreme point in a nondegenerate pivot. (Hint: 
Suppose you have a basic feasible solution x with basis B consisting of the first m 
columns of A. Further suppose that x, entered the basis. Consider the hyperplane 


passing through x, and whose normal vector is (p, p BIN+ (1, 1,..., 1, 0, 1,..., 1), 
where p is an arbitrary m—vector and the zero component of (1, 1, ..., 1, 0, 1,..., 1) 
appears at position k — m.) 


[5.56] Consider the collection of points satisfying x > 0 and Ax = b, where A is an 
m Xn matrix with rank m. Further suppose that the region is bounded. Let xg be 
an extreme point of the region, and let x,, x2,...,x, be its adjacent extreme points 


in the region (refer to Exercise 5.54). Let x be any point in the region. Show that x 
can be represented as 


k 
X=X9+ Y uj (X;—Xo), where w; 20, Vj=l,...,k. 
Jel 


0 
Interpret this result geometrically. [Hint: Let xo -|5"4|"(| (the nonbasic 


variables are placed first for convenience). Show that 


for j= 1,....k=—m where y; = B'a, and 


Aj = ign > 7 J 
I<ism | Vy 

[5.57] Suppose that the boundedness restriction in Exercise 5.56 is dropped. Can 

you generalize the foregoing result? Interpret your result geometrically. (Hint: 

Introduce the notion of an adjacent direction. Then x in the region can be 

represented as 
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X=Xot+ L uj (X;-Xo)t+ LT jd; 
jel jed 


where yu; 2 Ofor jelUJ, x j for 7 €/ are adjacent extreme points, and d j for 
j €J are adjacent extreme directions.) 


[5.58] Show that an extreme point of a bounded polyhedral set has a minimal 
objective if and only if it has an objective that is smaller than or equal to that of 
any adjacent extreme point. Can you generalize the result to the unbounded case? 
(Hint: Use Exercises 5.56 and 5.57.) 


[5.59] Given a basis B, consider an LU factorization such that PBQ = LU, where 
P and Q respectively permute the rows and columns of B, and where L and U are 
resultant lower and upper triangular factors. Show how each of the systems 
wB =cz for computing the simplex multiplier vector w, and By, =a, for 


determining the updated column of an entering variable x, can be solved via the 
solution of two appropriate triangular equation systems. 


NOTES AND REFERENCES 


1. The revised simplex method was devised by Dantzig and Orchard—Hays 
[1953] (also see Dantzig and Orchard—Hays [1954] for the product form of 
the inverse). For further reading on this topic, refer to Dantzig [1963a]. 


2: The experimentation related to fitting the regression equation Km nd? for 
computational effort in solving linear programs appears in Knolmayer 
[1982]. For information on theoretical bounds on computational effort using 
the simplex method, see Chapter 8. 

3. The simplex method using the LU-factorization of the basis was introduced 
by Bartels and Golub [1969] and is said to employ the “elimination form of 
the inverse.” For more information on this subject, see Gill et al. [1974] and 
Saunders [1976a, b]. For factorizations designed for use in a parallel 
computing environment, refer to Helgason et al. [1987]. In an alternative 
LU factorization approach, the rows and columns of B are suitably 
permuted (using permutation matrices P and Q), based on which the lower 
and upper triangular factors L and U, respectively, are computed such that 
PBQ = LU. In particular, when determining the simplex multiplier vector w 
via the system wB=cpz, this entails solving two triangular systems (see 


Exercise 5.59). However, Hu and Pan [2008] show how the simplex 
multiplier vector can be updated from one iteration to the next based on 
updates to the LU factors such that only one triangular system is solved. 
This yields a computational advantage, particularly for dense problems. 

4. The steepest edge variations of the rules for selecting entering variables are 
discussed in Harris [1975] and in Goldfarb and Reid [1977]. For problem 
scaling, see Kuhn and Quandt [1962], Tomlin [1975], and Sherali et al. 
[1983]. The concept of partial pricing or suboptimization discussed in 
Section 5.1 is standard and in widespread use. The particular description 
given here has been adapted from Frendewey and Glover's [1981] 
experiments on large-scale linear programs with embedded networks. Also, 
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see Gibbs et al. [1983]. For model and input/output management of large— 
scale linear programs, see Greenberg [1983] and Murphy and Stohr [1986]. 
Several modeling languages have been developed to enable a convenient 
input of a linear programming model into a solver, without requiring the 
user to manipulate the data into a particular packed—form representation as 
needed by the solver. Examples of commercial modeling languages include 
AMPL, OPL, GAMS, and LINGO. Examples of commercial solvers include 
CPLEX, OSL (see Forrest and Tomlin (1990)), and LINDO. (Refer to their 
respective online Web sites.) 

5. In Chapter 7 we describe the use of the revised simplex method in solving 
large-scale problems in the context of decomposition by generating 
columns at each iteration. 

6. The simplex method for bounded variables was developed by Dantzig 
[1955] at the RAND corporation to provide a shortcut routine for solving a 
problem of assigning personnel. The method was independently developed 
by Charnes and Lemke [1954]. 

7. The strongly feasible partition lexicographic cycling prevention rule for 
bounded variables linear programming problems was proposed by 
Cunningham [1976] and Murty [1978]. For further developments involving 
generalized upper bounds (GUBs) and variable upper bounds (VUBs), the 
interested reader is referred to Dantzig and Van Slyke [1965] and Schrage 
[1975, 1978]. 

8. The Karush-Kuhn-Tucker optimality conditions for nonlinear programs 
were developed independently by Karush [1939] in his master’s thesis, and 
by Kuhn and Tucker [1950]. A specialization of these conditions for linear 
programs is given in Section 5.4. For further reading on the theorems of the 
alternative and the KKT conditions, the reader may refer to Bazaraa, 
Sherali, and Shetty [2006], Mangasarian [1969], and Zangwil] [1969]. The 
KKT conditions for linear programming and the subject of duality are very 
closely associated. This fact will become apparent when the reader studies 
Chapter 6. 
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SIX: DUALITY AND SENSITIVITY 
ANALYSIS 


For every linear program we solve, there is another associated linear program 
that we happen to be simultaneously solving. This new linear program satisfies 
some very important properties. It may be used to obtain the solution to the 


original program, and its variables provide extremely useful information about 


This leads to rich economic interpretations related to the original linear 
programming problem. In fact, the roots of this problem lie in the characteriza- 
tion of the optimality conditions for the original linear program. For the sake of 
expository reference, we shall call the original linear programming problem the 
primal (linear programming) problem, and we shall call this related linear pro- 
gram the dual (linear programming) problem. Although the term “dual” comes 
from linear algebra, the term “primal” was suggested as an appropriate Latin 
antonym by Dantzig’s father, Tobias Dantzig (who was a mathematician), to 
substitute for the cumbersome phrase, “the original problem of which this is the 
dual.” Actually, the terms primal and dual for this related pair of linear pro- 
gramming problems are only relative, because the dual of the “dual” is the 
“primal” itself. 

We shall begin by formulating this new dual (linear programming) 
problem and proceed to develop some of its important properties. These 
properties will lead to two new algorithms, the dual simplex method and the 
primal—dual algorithm, for solving linear programs. Finally, we shall discuss the 
effect of variations in the data, that is, the cost coefficients, the right—hand-side 
coefficients, and the constraint coefficients, on the optimal solution to a linear 
program. 


6.1 FORMULATION OF THE DUAL PROBLEM 


Associated with each linear programming problem there is another linear pro- 
gramming problem called the dual. The dual linear program possesses many 
important properties relative to the original primal linear program. There are two 
important forms (definitions) of duality: the canonical form of duality and the 
standard form of duality. These two forms are completely equivalent. They arise 
respectively from the canonical and the standard representation of linear 
programming problems. 


Canonical Form of Duality 


Suppose that the primal linear program is given in the (canonical) form: 


P: Minimize ex 
subject to Ax 
x 


IV IV 
a 


Then the dual linear program is defined by: 
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D: Maximize wb 
subjectto wA < ¢ 

w > 0. 

Note that there is exactly one dual variable for each primal constraint and 


exactly one dual constraint for each primal variable. We shall say more about 
this later. 


Example 6.1 


Consider the following linear program and its dual: 


P: Minimize 6x, + 8x» 


subjectto 3x, + Xx, 2 4 
5x) + 2x, 2 7 
Xx, XxX, 2 0. 

D: Maximize 4m, + 7w 
subject to 3m, + S5w < 6 
w+ 2w < 8 
Ww, w, 2 0. 


Before proceeding further, try solving both problems and comparing their 
optimal objective values. This will provide a hint of things to come. 

In the canonical definition of duality it is important for Problem P to have 
a “minimization” objective with all “greater than or equal to” constraints and all 
“nonnegative” variables. In theory, to apply the canonical definition of duality 
we must first convert the primal linear program to the foregoing format. 
However, in practice it is possible to immediately write down the dual of any 
linear program. We shall discuss this shortly. 


Standard Form of Duality 


Another equivalent definition of duality may be given with the primal linear 
program stated in the following standard form: 


P: Minimize ex 


subjectto Ax = b 
x = 0 
Then the dual linear program is defined by: 
D: Maximize wb 
subjectto wA < ¢ 
w unrestricted. 


Example 6.2 


Consider the following linear program and its dual (compare this with Example 
6.1 above): 
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P: Minimize 6x, + 8x» 


subjectto 3x, + XxX, - x3 = 4 
5x, + 2x> — X4 = 7 
X> xX, X35 x4 2 0. 
D: Maximize 4m, + 7w, 
subjectto 3w, + 5wy < 6 
wot 2w, < 8 
—wW, < 0 
_ W2 < 0 
W> W> unrestricted. 


Given one of the definitions, canonical or standard, it is easy to demonstrate that 
the other definition is valid. For example, suppose that we accept the standard 
form as a definition and wish to demonstrate that the canonical form is correct. 
By adding slack variables to the canonical form of a linear program, we may 
apply the standard form of duality to obtain the dual problem. 


P: Minimize cx D: Maximize wb 
subject to Ax - Ix, = b subject to wA < c¢ 
X, x, 2 0. -wI < 0 
Ww unrestricted. 


But since —wI < 0 is the same as w > 0, we obtain the canonical form of the dual 
problem. 


Dual of the Dual 


Since the dual linear program is itself a linear program, we may wonder what its 
dual might be. Consider the dual in canonical form: 


Maximize wb 
subject to wA 


< 
we 


c 
0. 


Applying the transformation techniques of Chapter 1, we may rewrite this prob- 
lem in the form: 


Minimize (—-b’)w’ 
subject to (—A‘)w’ 


w 


(-c’) 
0 


VV IV 


The dual linear program for this linear program is given by (letting x’ play the 
role of the row vector of dual variables): 


Maximize x’ (-c’) 
subject to x‘(-A’) < (-b’) 
ers 0. 


But this is the same as: 
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P: Minimize ex 
subjectto Ax > b 
x > 0, 


which is precisely the original primal problem. Thus, we have the following 
lemma, which is known as the involutory property of duality. 


Lemma 6.1 


The dual of the dual is the primal. 


This lemma indicates that the definitions may be applied in reverse. The 
terms “primal” and “dual” are relative to the frame of reference we choose. 


Mixed Forms of Duality 


In practice, many linear programs contain some constraints of the “less than or 
equal to” type, some of the “greater than or equal to” type, and some of the 
“equal to” type. Also, variables may be “> 0,” “< 0,” or “unrestricted.” In 
theory, this presents no problem since we may apply the transformation 
techniques of Chapter 1 to convert any “mixed” problem to one of the primal or 
dual forms discussed, after which the dual can be readily obtained. However, 
such conversions can be tedious. Fortunately, it is not actually necessary to 
make these conversions, and it is possible to state immediately the dual of any 
linear program. 
Consider the following linear program: 


P: Minimize ¢)x;) + 9X. + €3x3 
subjectto Aj;X; + AjoX. + Aj3x3 2 Dd, 
AaiX} + Ag2X. + Ag3x3 < by 
A31X, + A32X2 + Ag3x%3 = bg 
x, 20, X. <0, = x3_sunrestricted. 


Converting this problem to canonical form by multiplying the second set 
of inequalities by —1, writing the equality constraint set equivalently as two 


inequalities, and substituting, x7 =—x5, x3 = x; — x3, we get: 
Minimize CX; — 9X2 + 3X3 — 3X3 
subject to =Ay,;x; — AyoX > + Ay3x3 - Ap3x3 2 Dd, 
~Agx, + AgxX. — Ag3x3 + An3x3 2 —by 
A3iX) — Ag2X2 + Ag3x3 — Ag3x3 2 D3 
~A31X, + Ag2X. — Ag3X3 + Ag3x3 2 —b3 
x, 20, x, 20, x3 20, x3 20. 


Denoting the dual variables associated with the four constraint sets as wy, 
W>, W3, and w3, respectively, we obtain the dual to this problem as follows: 
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Maximize wiyb; - Wb + wb; -  w3b; 
subject to wyAy; = WwyAd] + Ww3A3; _- w3A3; < Cy 
, ' " 
—WiAy. + WA n7 — W3A32 + W3A32 S Co 
‘ ‘ ” 
WiA}3, — W2Ao3 + W3A33 — W3A33 5 3 
‘ ‘ ” 
—W)Aj3, + W2A23 — W3A33 + W3A33 S C3 
w, 20, w> 20, w; 20, w3 20. 
Finally, using w. =~Ww» and w3 = wW3 — W3, the foregoing problem 


may be equivalently stated as follows: 


D: Maximize wb; + Woby + 3 W3b3 
subject to _w,A;; + WA; + W3A3; S$ ¢} 
WiAjz + W2Ao2 + W3A37 2 Cp 
WiAj3 + WA 3 + W3A33 = 63 
w, 20, w> <0, wy; unrestricted. 


Note how this form of the dual D relates to the original primal problem P, 
where W;, W>, and w3 are the dual variables associated with the three sets of 


constraints in P; similarly, x;, x2, and x3 are the “dual” variables associated 


with the three sets of constraints in D. Furthermore, note that the first set of 
constraints in P were already in the required form, and hence, w, > 0 in D. The 


second constraint set needed to be multiplied by —1 in order to be put in the 
required form, and hence, wy <0 in D. Similarly, because of the transformation 


needed on the third set of constraints in P, we obtain w3 as unrestricted in D. 


Likewise, the transformations needed on the variables in P to put it in the 
required canonical form dictate the form of the constraints in D. With these 
observations, the dual D to the problem P can now be written directly without 
using the intermediate transformation steps. These results are summarized in 
Table 6.1. In Exercise 6.1, we ask the reader to repeat this derivation using the 
standard form of duality. 


Example 6.3 
Consider the following linear program: 
Maximize 8x, + 3x, -— 2x3 
subjectto x, - 6x» + % 2 2 
5x, + 7X a 2x3 = -4 


x, <0, x, 20, x3 unrestricted. 


Applying the results of the table, we can immediately write down its dual: 
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Table 6.1 Relationships Between Primal and Dual Problems 


MINIMIZATION MAXIMIZATION 
g PROBLEM PROBLEM 2 
2 >0 <—_> < ‘Ss 
5 <0 <> 2 q 
> : : g 
Unrestricted <> = 
iS = <—_> 20 ‘ 
S aa — = 
Sy ra) 
oO = <> Unrestricted > 


Minimize 2w —- 4, 


subject to weo+ Sw < 8 
-6my, + Tw, 2 3 
a 2 W = —2 


w, <0, w) unrestricted. 


6.2 PRIMAL-DUAL RELATIONSHIPS 


The definition we have selected for the dual problem leads to many important 
relationships between the primal and dual linear programs. 


The Relationship Between Objective Values 


Consider the canonical form of duality and let xp and Wo be any feasible solu- 
tions to the primal and dual programs, respectively. Then Axg > b, Xo > 0, 
WoA <c, and Wo = 0. Multiplying Axg > b on the left by wy > 0 and woA <c 
on the right by xg > 0, we get 

CX 2 WoAXy 2 Wob. 
The result is the following. This is known as the weak duality property. 
Lemma 6.2 


The objective function value for any feasible solution to the minimization 
problem is always greater than or equal to the objective function value for any 
feasible solution to the maximization problem. In particular, the objective value 
of any feasible solution of the minimization problem gives an upper bound on 
the optimal objective of the maximization problem. Similarly, the objective 
value of any feasible solution of the maximization problem gives a lower bound 
on the optimal objective of the minimization problem. 

As an illustration of the application of this lemma, suppose that in 


Example 6.1 we select the feasible primal and dual solutions xg = (7/5,0)' and 
Wo = (2, 0). Then exg = 42/5 = 8.4 and wob = 8. Thus, the optimal solution 
for either problem has objective value between 8 and 8.4. This property can be 


Duality and Sensitivity Analysis 265 


invoked to terminate the solution of a linear programming problem with a near 
optimal solution. 
The following corollaries are immediate consequences of Lemma 6.2. 


Corollary 6.1 


If xg and wo are feasible solutions to the primal and dual problems, 
respectively, such that ¢xg = Wob, then xp and wg are optimal solutions to 
their respective problems. 


Corollary 6.2 


If either problem has an unbounded objective value, then the other problem 
possesses no feasible solution. 

Corollary 6.2 indicates that unboundedness in one problem implies 
infeasibility in the other problem. Is this property symmetric? Does infeasibility 
in one problem imply unboundedness in the other? The answer is, “Not 
necessarily.” This is best illustrated by the following example: 


Example 6.4 

Consider the following primal and dual problems: 
P: Minimize -x, — x 

subjectto x, -— x 2 | 

—X) + XQ > 1 

xX; x2 > 0. 
D: Maximize w+ wy 

subjectto we- w s< -l 

—Wwo+ Ww Ss -l 

Wi w, 2 0. 


On graphing both problems (in Figure 6.1), we find that neither problem 
possesses a feasible solution. 


Origin of Duality and the Karush-Kuhn—Tucker (KKT) 
Optimality Conditions 
Recall from Chapter 5 that the optimality conditions for a linear program state 


that a necessary and sufficient condition for x” to be an optimal point to the 
linear program: Minimize ex subject to Ax > b, x > 0 is that there exists a vector 


w’ such that 
1. Ax*>b,x" >0 
2. wA<c,w >0 
3. w'(Ax"—b) =0 
(c—w'A)x’ =0. 
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a 
Primal 


Mal 
Dual 


Figure 6.1. An example of infeasible primal and dual problems. 


Condition 1 simply requires that the optimal point x" must be feasible to the 
primal. In light of our discussion of duality, we can now interpret Condition 2. 


This condition indicates that the vector w" must be a feasible point for the dual 
problem. From complementary slackness (Condition 3), we find that ex” = 
w’b. Hence, w” must be an optimal solution to the dual problem. Consequently, 


the feasible region of the dual problem arises from Condition 2. Its objective 
function arises from the fact that any w feasible to the dual must satisfy wb < 


ex” by the weak duality property. Conditions 2 and 3 state, however, that the 
KKT solution provides a w” that is feasible to the dual and for which w"b = 


ex”. Hence, w” must maximize wb over the dual feasible region. Symmetrically, 
the KKT optimality conditions for the dual problem imply the existence of a 
primal feasible solution whose objective is equal to that of the optimal dual 
(why?). This leads to the following lemma, which is known as the strong duality 
property. 

Lemma 6.3 


If one problem possesses an optimal solution, then both problems possess 
optimal solutions and the two optimal objective values are equal. 


It is also possible to see how the dual problem arises naturally in the 
context of the simplex algorithm. Each tableau in canonical form represents a 
situation in which some multiples w,,..., w,, of the m rows in Ax — x, = b have 
been added to the objective row, where x, > 0 is the vector of slack variables. 
This means that the objective row coefficients for the x—variables are —c + wA, 
those for the x, variables are —w, and the right-hand-side is wb. Note that dual 
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feasibility (Condition 2) requires a set of these so-called simplex multipliers w* 
such that the canonical objective row coefficients are nonpositive for all the x— 
and x, —variables. Indeed, if the primal problem has been solved to optimality, 


then the available simplex multipliers w* = ¢ 3B: where B is the optimal basis 


matrix, satisfy these dual feasibility conditions. Moreover, if x” is a primal 


optimal solution, its objective value ex” coincides with w’b, the right-hand— 


side value in the objective row of the optimal tableau. In view of the weak 
duality property of Lemma 6.2, we have that, at optimality, the simplex 


multipliers w* must be a maximizing solution to wb subject to the constraints 
wA <c and w > 0, that is, to the so-called dual linear programming problem. 


Lemma 6.3 provides an insightful result that is related to Corollary 6.2. 
For the linear program P: Minimize ex, subject to Ax > b, x > 0, consider the 
homogeneous form of its dual problem, namely, HD: Maximize wb, subject to 
wA < 0, w > 0. Note that the dual to HD has the same feasible region as P. 
Hence, if HD is unbounded, then by Corollary 6.2, P is infeasible. Conversely, if 
P is infeasible, then HD must be unbounded, or else, since HD is feasible (w = 0 
is a feasible solution), it must have an optimal solution. But Lemma 6.3 tells us 
that the dual to HD then has an optimal solution; that is, P is feasible, a contra- 
diction. Therefore, we have the following result. We ask the reader to verify this 
using Example 6.4. It is insightful to note that the homogeneous form of the dual 
arises from the dual problem via a characterization of its recession directions as 
in Chapter 2. 


Corollary 6.3 


The primal problem is infeasible if and only if the homogeneous form of the 
dual problem is unbounded (and vice versa). 


By utilizing the foregoing results we obtain two important basic theorems 
of duality. These two theorems will permit us to use the dual problem to solve 
the primal problem and, also, to develop new algorithms to solve both problems. 


The Fundamental Theorem of Duality 


Combining the results of the lemmas, corollaries, and examples of the previous 
section we obtain the following: 


Theorem 6.1 (Fundamental Theorem of Duality) 


With regard to the primal and dual linear programming problems, exactly one of 
the following statements is true: 


1. Both possess optimal solutions x” and w* with ex” = w'b. 

2. One problem has an unbounded optimal objective value, in which 
case the other problem must be infeasible. 

3. Both problems are infeasible. 
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From this theorem we see that duality is not completely symmetric. The 
best we can say is that (here optimal means having a finite optimum, and un- 
bounded means having an unbounded optimal objective value): 


P OPTIMAL &  D_ OPTIMAL 

P(D) UNBOUNDED =  D(P) INFEASIBLE 

P(D) INFEASIBLE =  D(P) UNBOUNDED OR INFEASIBLE 
= 


P(D) INFEASIBLE D(P) UNBOUNDED IN 
HOMOGENEOUS FORM 


Complementary Slackness and the Supervisor’s Principle 


We have seen that if x and w are any pair of primal and dual feasible solutions 
to the linear program: Minimize ex, subject to Ax > b, x > 0, then they are both 
optimal to their respective problems if and only if they have the same objective 
value (that is, ex = wb). This is known as the supervisor’s principle because it 
provides a simple check by which a “supervisor” can verify the (simultaneous) 
optimality of a pair of candidate primal and dual feasible solutions. The KKT 
conditions tell us that another equivalent supervisor’s principle is to check for 


the complementary slackness conditions (written as Condition 3). Indeed, if x* 
and w" respectively satisfy primal and dual feasibility (Conditions 1 and 2), 
then ex” = w’b, that is, w"(Ax" — b) + (c — w’A)x" = 0 holds true, if and only 
if Condition 3 is satisfied, since w* > 0, Ax” —b > 0,c— w*A > 0, and x* > 0. 
In expanded form, since w* > 0 and Ax” — b > 0, then w*(Ax* — b) = 0 implies 
w} (a'x* — b;) = 0 for i = 1,..., m. Similarly, (¢e — w*A)x" = 0 implies (er 
w'a;)x; = 0 for =1,..., 7. 
Thus, we have the following theorem: 
Theorem 6.2 (Complementary Slackness Theorem) 
Let x* and w” be any feasible solutions to the primal and dual problems in the 
canonical form. Then they are respectively optimal if and only if 
(c; —w'a x} =0, j=l... 
and 
w; (a'x” —b,)=0, i=l,...,m. 
This is a very important theorem relating the primal and dual problems. It 


obviously indicates that at least one of the two terms in each expression must be 
zero. In particular, 


“>0>w'a,= 
xj wa;=c; 
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* * 
Ww aj <Cj => x; =0 


we >O0=a'x" =); 


a'x* >b; => Ww} =0, 


Hence, at optimality, if a variable in one problem is positive, then the corre- 
sponding constraint in the other problem must be tight. If a constraint in one 
problem is not tight, then the corresponding variable in the other problem must 
be zero. 

Suppose that we let x,,; =a'x—b;, >0, i = 1,...,m, be the m slack vari- 
ables in the primal problem and let w,,, ; =c;—-wa;20,j=1,...,”, be then 
slack variables in the dual problem (in Chapter 5 while stating the KKT 
conditions, w,,, ; was denoted by v;). Then we may rewrite the complementary 


slackness conditions as follows: 


* * . 
XiWans j =0, j=l, 


* . 
Wj Xn4i =0, i=l,....m. 


This relates variables in one problem to slack variables in the other problem, In 
particular, x; and w,,, ; forj = 1, ..., 2, and similarly, x,,,; and w; for i= 1,...,m, 


are known as complementary pairs of variables. 
Using the Dual to Solve the Primal Problem 


We now have at hand some powerful analytical tools, in the form of the two 
theorems of this section, to utilize the dual problem in solving the primal 
problem. Let us illustrate this potential usefulness by the following example: 


Example 6.5 
Consider the following primal and dual problems: 


P: Minimize 2x1 i 3x9 + 5x3 + 2X4 + 3X5 


subject to Xo Ft XQ + 2x3 + xXg + 3X5 > 4 
2x, — 2x. + 3x3; + X4 + x5 2 3 
X> X25 x3, x4, xy 2 0. 
D: Maximize 4w, + 3w 

subjectto w, + 2w, < 2 

Ww - 2w < 3 

2W aE 3w s 5 

w+ WW < 2 

3m, + WwW S 3 

Wi, Ww, 2 0. 
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Figure 6.2. Solving the dual problem graphically. 
Since the dual has only two variables, we may solve it graphically as shown in 


Figure 6.2. The optimal solution to the dual is w; = 4/5, w) = 3/5 with objec- 


tive value 5. Right away we know that z* = 5. Utilizing the theorem of 
complementary slackness, we further know that x = x3 = x4 = 0 since none 


of the corresponding complementary dual constraints are tight. Since wy, w> > 
0, then x; + 3x3 =4 and 2x; + x5 =3. From these two equations we get x; = | 
and xs = 1. Thus, we have obtained a primal optimal solution by using the 
duality theorems and the foregoing dual optimal solution. 

6.3 ECONOMIC INTERPRETATION OF THE DUAL 


Consider the following linear program and its dual: 


P: Minimize ex D: Maximize wb 
subject to Ax = b subjectto wA < ¢ (6.1) 
x = 0. we 0. 


Let B be an optimal basis for the primal problem with ¢p as its associated cost 


vector. Suppose that this optimal basic feasible solution x” is nondegenerate. 
Denoting J as the index set for the nonbasic variables x;, which may include 


both slack and structural variables (that is, J Cc {I,....2+m}), we have 
z=¢gB ‘b- ¥ (z;-c;)x;=w'b— ¥ (z;-c,)x;. (6.2) 
jet Jed 


If the ith right-hand-side 5, is perturbed slightly (positively or negatively), 
since the current basic feasible solution remains feasible, we maintain its 
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optimality. Hence, if z* is the optimal objective function value and By! is the 


ith column of Bo, we have 
; =. (6.3) 


Thus, w; is the rate of change of the optimal objective value with a unit 


I 
increase in the ith right-hand-side value, given that the current nonbasic vari- 
ables are held at zero (regardless of feasibility). Because w; > 0, z* will increase 


(respectively, decrease) or stay constant as b; increases (respectively, decreases). 


Economically, we may think of w" as a vector of shadow prices for the 
right-hand-side vector. To illustrate, if the ith constraint represents a demand 
for production of at least }; units of the ith product and ex represents the total 


cost of production, then w; is the incremental cost of producing one more unit 


of the ith product. Put another way, w; is the fair price we would pay to have 


an extra unit of the ith product. 

We may also interpret the entire dual problem economically. Suppose 
that we engage a firm to produce specified amounts },, 5,...,5,, of m outputs or 
goods. The firm may engage in any of 7 activities at varying levels to produce 
the outputs. Each activity j has its own unit cost c;, and we agree to pay the 


total cost of production. From our point of view, we would like to have control 
over the firm's operations so that we can specify the mix and levels of activities 
that the firm will engage in so as to minimize the total production cost. If ayy 


denotes the amount of product i generated by one unit of activity j, then 
Liel a,x; represents the units of output i that are produced. These must be 


greater than or equal to the required amount b;. Therefore, we wish to solve the 
following problem, which is precisely the primal problem: 


n 
Minimize > c 


Xs 
jal ae) 
F n 
subjectto > ayX; = b,, i=1,...,m 
j=l 
xj 2 0, JH)... 


Instead of trying to contro] the operation of the firm to obtain the most desirable 
mix of activities, suppose that we agree to pay the firm unit prices w,, W9,...,W,, 


for each of the m outputs. However, we also stipulate that these prices 
announced by the firm must be fair. Since aj is the number of units of output i 


produced by one unit of activity j and w, is the unit price of output i, then 
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27214 can be interpreted as the unit price of activity j consistent with the 
prices w,, W,...,W,,. Therefore, we tell the firm that the implicit price of activity 


j, namely 27", a;;w;, should not exceed the actual cost c;. Therefore, the firm 


must observe the constraints 3)7";a;w; Sc; for j = 1,...,2. Within these con- 


straints the firm would like to choose a set of prices that maximizes its return 
i, ¥;5;. This leads to the following dual problem considered by the firm: 


m 
Maximize > w,b; 
i=] 


m 

subjectto Yasw, < c¢ 
i=] 

> 0, 1=1,...,m. 


Wj 


The main duality theorem states that, provided an optimum exists, there is an 
equilibrium set of activities and set of prices where the minimal production cost 
is equal to the maximal return. That the two objectives are equal at optimality 
becomes intuitively clear by noting that they represent the fair charge to the 
customer, where the primal objective is derived by cost considerations and the 
dual objective is arrived at by a pricing mechanism. These shadow prices are 
often used by regulated industries, such as electric utilities, to set and justify 
prices. 

Observe that the complementary slackness theorem says that if, at opti- 


mality, eel ayX; >b;, then w; = 0, that is, if the optimal level of activities 


that meets all demand requirements automatically produces an excess of product 
i, then the incremental cost associated with marginally increasing 5, is naturally 
j> then x = 0, that is, if the total revenue 
generated via the items produced by a unit level of activity j is less than the 
associated production cost, then the level of activity 7 should be zero at 
optimality. 

Interchanging the roles of the previous primal and dual problems, there is 
another interesting economic interpretation that can be given. Consider the 
following pair of dual problems, where the problems D and P have been 
rewritten as P and D, respectively: 


zero. Similarly, if D7, ayw; <c 


n 
P: Maximize 2 CjxX; 
j=l 


lA 
o 


n 
subject to a yjX; 


x 2 0. 
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m 
D: Minimize > 4, 
= 


m 
subject to 2 ayy Bes, for j =1,...,.7 
i= 
we 0. 
Assume that problem P is a product—mix problem in which n products are being 
manufactured using some m types of resources. Here, x; represents the number 
of units (per year) produced of product, and 5; represents the number of units 
available for resource i for j= 1, ..., 2 andi=1,...,m. A unit of product / fetches 
a profit of c F units and consumes aij units of resource i fori= 1, ..., m. 
As before, let w; denote the shadow price or the imputed value or the 


fair market price/rent for a unit of resource 7. Then the dual constraints have the 
following interpretation. Observe that if the manufacturer considers renting out 
the m resources at unit prices w,,...,W,, instead of manufacturing the mix of 
products, then every unit of product 7 not manufactured would result in a loss of 
profit of c; units but would earn a rent of L/",a,;w, units by virtue of the 
resources this releases. The dual constraints ensure that this venture is at least as 
profitable as manufacturing the products. However, in order to maintain fair or 
competitive prices while guaranteeing this profitability, the dual objective seeks 
to minimize the total rent. The main duality result tells us that, provided an 
optimum exists, the resulting total rent will match the total optimal profit 
obtainable by manufacturing the products. Hence, w; imputes a value that the 


manufacturer ascribes to a unit of the ith resource from the viewpoint of 
optimally operating the production process. In particular, at optimality, the 
complementary slackness theorem says that if not all available units of resource 


i are utilized, then the marginal worth wW} of an extra unit of resource / is 


naturally zero. Similarly, if the net value 2)", ay W; of the resources consumed 
by a unit level of production of product / exceeds the realizable profit c;, then 


the optimal production level of product / should be zero. 


Shadow Prices Under Degeneracy 


Now, consider the pair of primal and dual problems given in Equation (6.1), and 
assume that we have a degenerate optimal basic feasible solution. Then, while 


the derivative of z in Equation (6.2) with respect to b; remains we in the usual 
sense of the derivative where all other parameters b,, k#i, and x . JéJ, 
are held constant at their present values, this may not reflect the true “shadow 


price.” That is, Equation (6.3) may not be valid. In fact, as will be seen later, z” 
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Figure 6.3. Variation in z* as a function of 5, . 


may not be a differentiable function of 5;. In other words, a perturbation in b; 


may make the current optimal basis infeasible and a change in basis may be 
required. 
To be more precise, let us assume that D in Equation (6.1) is feasible so 


that P is not unbounded for any value of b. Let us denote by z (b;) the optimal 
objective function value as a function of the right-hand-side (parameter) ,. 
Then, as long as 5; varies in a region that leaves P feasible, by the extreme point 


optimality property and the duality theorem, we have z’(b;) = maximum {wi b: 


j=,...,E}, where w/, j = 1....,E, are the extreme points of the dual feasible 
region. Note that this admissible region for b;, which leaves P feasible, is a 


polyhedral set F ¢ R, given by b’d <0 for all extreme directions d of the dual 
feasible region (why?). Denoting Laeivi dy as we for j = 1,...,.£, we can 
rewrite z'(b;) as follows: 


z (bh) = maximum {wd + wb; :jH=l,..., BE}. (6.4) 


Observe that z'(b;) is the pointwise maximum of a finite number of 
affine functions in the variable b;, as long as b; ¢ F. Hence, as is shown in 


. * . . . . . . 
Figure 6.3, z (b;) is piecewise linear and convex over F. Also, since an increase 


in b; makes the feasible region of P more restrictive, z°(b;) is also nondecreas- 


ing as 5; increases. 


Referring to Figure 6.3 and assuming that z (-) is composed of four 
affine functions with the associated dual extreme points as shown in the figure, 
consider ; = f;. At this point, z°(b;) is differentiable, and in fact, 0z°/0b; = 
wr, which agrees with Equation (6.3). However, when b; = £5, for example, 


z'(b;) has a kink at this point and is nondifferentiable. Note that when b; = J», 
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the dual has alternative optimal extreme point solutions w, w, and w*. This 
implies degeneracy at optimality in the primal problem, for if an optimal primal 
basic feasible solution x” with basis B is nondegenerate, then by the comple- 


mentary slackness theorem, any dual optimal solution w must satisfy the dual 
constraints with respect to the basic variable columns as equalities, that is, wB = 


cp must hold true. This gives w = ¢ ,B! as the unique dual optimum, a con- 
tradiction. In fact, since any optimal dual extreme point solution must be com- 
plementary to x”, the dual constraints corresponding to the g < m positive basic 
variables in x" must be binding. This gives q linearly independent binding 
hyperplanes at optimality in the dual from the (m + n) inequalities defining the 
dual feasible region. Each choice of (m — q) additional (linearly independent) 
hyperplanes that gives some optimal dual extreme point corresponds to a choice 
of the (m — g) columns of the variables (structural or slacks) in P associated 


with these constraints. Therefore, because of the linear independence, this 
corresponds to a choice of a basis in P, and since the nonbasic variables for this 


. . * . . . . . * . 
basis are zero in x , this is a basis associated with x . Hence, the alternative 
optimal dual extreme point solutions all correspond to alternative bases associ- 


ated with the primal degenerate vertex x". 
Now, from Equation (6.4), we can define the one-sided (right-hand and 


left-hand) directional derivatives of z*(b;) with respect to b,. The right-hand 


derivative, denoted 4*z*/db,, gives the rate of change in z” with an increase in 
b;. As evident from Equation (6.4) (see Figure 6.3), so long as P remains 
feasible as 5; increases marginally (else, the right-hand shadow price is infi- 
nite), we have 
orz 


ab, 


= maximum{w/ :-w/ isan optimal dual vertex at 5;}. (6.5) 


Similarly, the left-hand derivative, denoted 6” z"/0b;, gives the negative of the 


rate of change in z” as b; decreases. Again, from Equation (6.4), referring to 
Figure 6.3, this is given by 

oz" 

Ob 


7 


= minimum{w/ :w/ isan optimal dual vertex at 5;}. (6.6) 


Equations (6.5) and (6.6) respectively give the right-hand and left-hand shadow 
prices associated with the ith right-hand-side. Observe that if x" is nondegener- 
ate, so that there is a unique dual optimal extreme point w’, we get 0° z"/ab, = 


dz" /éb; = w;, as in Equation (6.3). Later, we shall comment on how these 


right-hand and left-hand shadow prices can be easily computed in general via a 
parametric analysis. 
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Example 6.6 

Consider the (generic) instance of Problem P of Equation (6.1) as depicted in 
Figure 6.4. Here, m = 3; n =2; x3, x4, and x5 are the slack variables associated 
with the three primal constraints, and A,, Aj, and A; denote the gradients of 


the three rows of A. Observe that since the feasible half-spaces are Ax > b, the 
normal vectors — A}, — Aj, and —A3 are as shown in the figure. Note that each 


dual basis has two basic variables. 
First, consider Figure 6.4a. This problem has a unique primal optimal 


solution x". The KKT conditions at x" require —c to be written as -wA, ~ 


w A>, and hence, w, and wy are both positive and are uniquely determined. 


That is, there is a unique dual extreme point solution given by w" = “e Ps ad 
(Here, w, and w, are the basic variables, and w3, w4, and ws are nonbasic, 
where wy and ws are the slack variables for the two dual constraints.) Because 
the maximum in Equation (6.4) is unique, Equation (6.3) holds and we have 
dz*/0b, = we > 0, 6z"/Aby = w3 > 0, and éz"/Ab; = w} = 0. It is instructive to 
note that in the optimal tableau representing x", 6z°/Ox, = — (z3-c3) = wy. 
Note that the edge along which we move when we hold x4 = 0 and increase x3 
is precisely the direction in which the optimal solution moves when 5, is 


(marginally) increased. 
Second, in Figure 6.4a, rotate —¢ so that it aligns with the vector —A,. We 


now have alternative optimal primal solutions, one of which is degenerate. 
However, since x” is still optimal, and since by the KKT conditions, all optimal 


dual solutions must be complementary to x", we only need to examine any one 
primal optimal solution, say, x", in order to capture a// dual optimal solutions. 
Applying the KKT conditions at x", we find that again there is only one dual 


optimal solution w" where w; > 0 is a solution to— w,A, =~—e, and w=; =0. 


Hence, Equation (6.3) again holds true, although the primal problem has a 
degenerate optimum. 


Finally, consider Figure 6.4b, where x" is the unique optimum. 
However, it is degenerate and there are two optimal bases, having either x3 and 


Xs as nonbasic variables, or x4 and xs as nonbasic variables. (Note that not all 


z;—c; <0 for the choice of nonbasic variables x3 and x4; in fact, for this 


choice, 24 —c4 > 0 (why?).) Hence, each of these two optimal bases gives a dual 


optimal extreme point. The first basis gives a dual optimal vertex w! in 
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x2 ~A> 


Case (a) Case (6) 
Figure 6.4. Illustration of shadow prices. 


which w, and wy are basic, with wi > 0, wi > 0, and wh = 0. The second basis 
gives a dual optimal vertex w” in which wy and w, are basic, with w7 = 0, wy > 
0, and ws > 0. In the first case, —c is written (uniquely) in terms of A; and —A 
as —wiAy = whA3, and in the second case, —c is written (uniquely) in terms of 
—A, and —Ag as ~ we A> — Weg. By completing the parallelogram in Figure 


6.4b to compute the foregoing vector representations, note that wh < we : 
Hence, from Equations (6.5) and (6.6), we get 


otz Oz 2 a*z 


Oz 1 
—=w >0, —= =0, == = y% >0, —=w, =0, 
ab Wi ab, mi aby 2 aby 2 
and 
+ — 
ie wy >0 and is greater than a wh >0. 
ab; Ob; 


We ask the reader to verify these right-hand and left-hand shadow prices by 
actually perturbing the respective constraints in Figure 6.4b in the appropriate 


directions. (For example, to realize 0°z/0b, and 0” z/éb,, perturb the first con- 
straint parallel to itself in the direction of A, and —Aj, respectively.) In Exercise 
6.21, we ask the reader to repeat the analysis with —c aligned along — A). 


6.4 THE DUAL SIMPLEX METHOD 


In this section, we describe the dual simplex method, which solves the dual 
problem directly on the (primal) simplex tableau. At each iteration we move 
from a basic feasible solution of the dual problem to an improved basic feasible 
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solution until optimality of the dual (and also the primal) is reached, or else, 
until we conclude that the dual is unbounded and that the primal is infeasible. 


Interpretation of Dual Feasibility on the Primal Simplex Tableau 


Consider the following linear programming problem: 
Minimize ex 
subject to Ax > b 
x > 0. 
Let B be a basis that is not necessarily feasible and consider the following 
tableau: 


SLACK VARIABLES 


The tableau represents a primal feasible solution if b; > 0 for i = 1,..., m; that is, 
ifb=B'b>0. Furthermore, the tableau is optimal if zpr-ejs 0 for j = 1,..., 7 
+m. Define w= cpB. For j = l,..., 2, we have 

= -1 = = 
z;—C; =¢pB aj;—c; =Wwa; Cj 


Hence, zp-cj S 0 for j = 1,..., 2 implies that wa;—c; < 0 for j = 1,..., 7, 


J 
which in turn implies that wA < c. Furthermore, note that a,,,; =—e; and c,4; = 
0 for i= 1,..., m, and so we have 
Znti —Cnti = WAnti — nti 
= w(-e;)—0 
= -w i=1,...,m. 


In addition, if z,,;-—C,,; <0 for i=1,..., m, then w; 20 for i= 1.,..., m, and so, 
w 2 0. We have just shown that z;—c; <0 forj=1,....n+m implies that wA 
< cand w> 0, where w= c ,Bl. In other words, dual feasibility is precisely 
the simplex optimality criteria Ze CpS 0 for all j. At optimality, w* = ¢pB"! 
and the dual objective value wb = (cgB!)b =Cp (B'b) = epb = z’, that is, 


the optimal primal and dual objective values are equal. Thus, we have the 
following result: 
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Lemma 6.4 


At optimality of the primal minimization problem in the canonical form (that is, 


zjp-c7 0 for all j), w’ =e zB! is an optimal solution to the dual problem. 


Furthermore, w; = -(Zn4j) —Cn+i) =—Zn4; for i=1,...,m. 
Note that a symmetric analysis holds for the equality constrained case. 


For an optimal tableau with basis B, taking w’ = ¢ BBo we again have that the 


primal optimality conditions zj-e; <0 imply wa, —c, $0 forj=1,..., m, that 


is, w’ is dual feasible. Moreover, the optimal primal objective value z* equals 
cpB'b = wb, and so the solution w’ = cpB! is optimal to the dual 
problem. 


The Dual Simplex Method 
Consider the following linear programming problem: 


Minimize cx 
subjectto Ax = b 
x 2 0. 
In certain instances it is difficult to find a starting basic solution that is 
feasible (that is, all b; > 0) to a linear program without adding artificial 


variables. In these same instances it might be possible to find a starting basis, 
which is not necessarily feasible, but that is dual feasible (that is, all z 77S <0 


for a minimization problem). In such cases it is useful to develop a variant of the 
simplex method that would produce a series of simplex tableaux that maintain 
dual feasibility and complementary slackness and strive toward primal 
feasibility. 


Consider the tableau representing a basic solution at some iteration. 


Suppose that the tableau is dual feasible (that is, z j7ej <0 for a minimization 


problem). If the tableau is also primal feasible (that is, all b; > 0), then we have 
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an optimal solution. Otherwise, consider some b, < 0. By selecting row r as a 
pivot row and some column k such that y,, < 0 as a pivot column, we can make 
the new right—hand-side by > 0. Through a series of such pivots we hope to 
make all b; > 0 while maintaining all zj-cj <9, and thus achieve optimality. 


The question that remains is how do we select the pivot column so as to 
maintain dual feasibility after pivoting. The pivot column & is determined by the 
following minimum ratio test: 


Zz Z2-—-C:; 
7k = Sk ~ minimums tsy, < Ob. (6.7) 
; y 
rk dri 


Note that the new entries in row 0 after pivoting are given by 
, Yrj 
(z; —c;) = (2; “C= — Cx). 
rk 


If dy 2 0, and since z, —c, < 0 and y,, < 0, then (Vj! Vee ZK —c,) = 0, and 


Gee Since the previous solution was dual feasible, then 


z,-¢; $0, and hence (z; —c p) < 0. Next, consider the case where y,; < 0. By 


Equation (6.1) we have 


i 
--c.)/ < 
hence (2; c;) 4 


Spe oop ad 
Yrk dy 
Multiplying both sides by Vij <0, we get z; —c; — (Yy/Va ZK — CK) <9, that 
is, (zj-c;)' < 0. To summarize, if the pivot column is chosen according to 
Equation (6.7), then the new basis obtained by pivoting at y,, is still dual 
feasible. Moreover, the dual objective after pivoting is given by cpB'b = 
(z, — Cy )b,/ 4. Since z, —cy <0, b. <0, and y,, < 0, then -(z, —c,)b,/y,~ 20 
and the dual objective improves over the current value of c pB! b= wb. 


We have just described a procedure that moves from a dual basic feasible 
solution to an improved (at least not worse) basic dual feasible solution. To 
complete the analysis we must consider the case when y,; 2 0 for all j, and 


hence no column is eligible to be the pivot column. In this case the ith row 
reads: ¥ ; y,;x; = ,. Since y,; 2 0 for all /, and x; is required to be nonnegative, 


then >; pIyjr; 2 0 for any feasible solution. However, 5, <0. This contradiction 


shows that the primal is infeasible, and so, the dual is unbounded (why?). In 
Exercise 6.38 we ask the reader to show directly that the dual is unbounded by 
constructing a direction satisfying the unboundedness criterion. 
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Summary of the Dual Simplex Method (Minimization Problem) 


INITIALIZATION STEP 


Find a basis B of the primal such that z; ac; = c,B ‘a, — ce; $0 for all 7 (in 
Section 6.6 we describe a procedure for finding such a basis if it is not immedi- 
ately available). 


MAIN STEP 


l. Ifb =B'b > 0, stop; the current solution is optimal. Otherwise, 
select a pivot row r with b, <0; say, 5, = minimum {6;}. 
2. If y,; 2 0 for all j, stop; the dual is unbounded and the primal is 


infeasible. Otherwise, select the pivot column & by the following 
minimum ratio test: 


2.-C aa ZC 
kk — minimums —— Yq <0 i 
Yrk J 


Vrj 
3. Pivot at y,, and return to Step I. 


In Exercise 6.62, we ask the reader to develop a similar algorithm for the 
bounded variables case. 


Example 6.7 


Consider the following problem: 
Minimize 2x, + 3x, + 4x3 


subjectto x + 2x, + x3; 2 3 
2x, - XX. + 3x3 2 4 
X)> XxX, X3 > 0. 


A starting basic solution that is dual feasible can be obtained by utilizing the 
slack variables x, and x5. This results from the fact that the cost vector is 


nonnegative. Applying the dual simplex method, we obtain the following series 
of tableaux: 
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X3 X4 Xs RHS 


Zz xX x 
1 


2 
0 
l 
0 


; " 
x 1 7/5 -1/5 -2/5 | 11/5 


Because b > 0 and z pW Cis 0 for all 7, an optimal pair of primal and dual solu- 


tions are at hand. In particular, 
(xp ,7,%3,.x4,%5) =(11/5,2/5,0,0,0), and = (wy, w5) = (8/5,1/5). 
Note that w; and W> are, respectively, the negatives of the z j ~¢; entries under 


the slack variables x4 and x5. Also, note that in each subsequent tableau, the 


value of the objective function is increasing, as it should, while solving the dual 
(maximization) problem, given nondegenerate pivots. 


The Complementary Basic Dual Solution 


Consider the pair of primal and dual linear programs in canonical form, where A 
ism xn. 


P: Minimize ex D: Maximize wb 
subject to Ax = b subject to wA < ¢ 
x 2 0. we 0. 


Recall that the variables x, ...,x,, in the primal are complementary to the 


slack variables w,,,1,....Wm+n in the dual, and the slack variables x,.,),.. 


2°’ m+n Xnim 


in the primal are complementary to the variables w,,...,w,, in the dual. Now, 
consider any basis B for the primal problem (feasible or not). The corresponding 
tableau for this basis has (w, ...,W,,) = cpB! as the simplex multipliers that 
have been used with the constraint rows when adding them to the objective row. 
Accordingly, for this solution w, the (z j ~¢;)-values for the x ;—variables are 


wa; — C; =—Wy4; forj=1,...,m, and the z,,; — Cy4; values for the x,4;- 


variables are —w, for i= 1, ..., m, as seen at the beginning of this section. Now, 


Ze s= 0 for x; basic, je {l,....2+m}, and in fact, making these Bae} 


equal to zero uniquely determines the (z;-—c;)—values for the nonbasic 


variables. In other words, putting the dual variables complementary to the basic 


x;-variables equal to zero, uniquely determines from the dual equality system 


the other dual variables that are complementary to the nonbasic x j~variables. 


Hence, the dual solution, the negative of which appears in the objective row of 
the current tableau, is a basic dual solution. The basic dual variables are 
complementary to the nonbasic primal variables, and correspondingly, the 
nonbasic dual variables are complementary to the basic primal variables. Hence, 
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letting xgg and Xgj be the basic and nonbasic variables, respectively, from 
the variables x,,...,x, and letting xgg and xg, be the basic and nonbasic 
variables, respectively, from the slack variables (X41, ----Xpim), We Can rewrite 


the primal constraints as 
Ai1X0B + Aj2Xon —Xsy = by 
Ax1XoB + A22Xon — Xsp = bp. (6.8) 


Then the primal basis B is given by the columns of xgg and Xgg as 


XOB XSB 

A 0 6.9 
B=) 11 } (6.9) 

A> -I 


Now, denote by Woz and Woy the dual variables associated with the two con- 
straint sets in Equation (6.8). Let wsy and Wsp be the slack variables in the 
dual constraints written with respect to the columns of the variables xpg and 
Xon, respectively. Then, with ¢ partitioned accordingly as (egg, Coy), the 
dual constraints can be written as follows: 


t t t t t t 
AiiWog + A21Won +Wsw = CoB 


t t t t t t 
Ai2Wop + A2Won +Wsp =Con- 


Hence, the complementary dual basis associated with the primal basis B has 
Wop and weg as the basic variables and is given by 


Wop Wsp 
Ai; | 0 (6.10) 
Ab I 


We ask the reader to verify in Exercise 6.39 that the matrix (6.10) is 
invertible given that B™! in Equation (6.9) exists, and in fact, the dual basic 
solution corresponding to Equation (6.10) gives (w,..., Wy) = cpB |, where Cz 
= (Cop. 0). Note that since the basic variables in one problem are complemen- 


tary to the nonbasic variables in the other problem, the complementary slackness 
condition holds for this primal—dual pair of basic solutions. 

Next, consider the following pair of primal and dual problems in standard 
form: 


P: Minimize cx D: Maximize wb 
subject to Ax = b subjectto wA < c 
x20 Ww unrestricted. 
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Given any primal basis B, there is, as previously, an associated complementary 
dual basis. To illustrate, introduce the dual slack vector w, so that wA + w, = 


c. The dual constraints can be rewritten in the following more convenient form: 
A‘w' +Iw, =e! 
w’ unrestricted 


t 
w, 20. 


Given the primal basis B, recall that w = cpB Substituting in the dual con- 
straints, we get 


we = AW 
= oc -A(Bl)' ch 
= i? ; (Jae (6.11) 
= es _N! eb. } 


Note that w = ¢,B™! and Equation (6.11) lead naturally to a dual basis. Since 


both ¢ ,B! and ¢4, —N’ (Bly! cp are not necessarily zero, then the vector w 
and the last n — m components of w,, which are complementary to the primal 
nonbasic variables, form the dual basis. In particular, the dual basis correspond- 
ing to the primal basis B is given by 


B’ 0 


The rank of the preceding matrix is n. The primal basis is feasible if Bb >0 
and the dual basis is feasible if w, > 0; that is, if ey — cpB'N = 0 (See 
Equation (6.11)). Even if these conditions do not hold true, the primal and dual 


bases are complementary in the sense that the complementary slackness 
condition (WA — c)x = 0 is satisfied, because 


-1 
(wA—c)x = w,x = (0,¢y ~<a) _ =: 


To summarize, during any dual simplex iteration, we have a primal basis that is 
not necessarily feasible and a complementary dual feasible basis. At termina- 
tion, primal feasibility is attained, and so all the KKT optimality conditions hold 
true. 
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Finite Convergence of the Dual Simplex Method 


First, let us establish finite convergence in the absence of dual degeneracy. Note 
that the dual simplex method moves among dual feasible bases. Also, recall that 
the difference in the dual objective value between two successive iterations 1s 


~(z, —c; )b,/y,4. Note that b. < 0, yp < 0, and z,—cy, < 0, and hence, 


~(z4 —Cy,)b,/y,4 2 0. In particular, if z, —cz <0, then the dual objective strictly 
increases, and hence no basis can be repeated and the algorithm must converge 
in a finite number of steps. By the foregoing characterization of the 
complementary dual basis, and because x, is a nonbasic primal variable, then 


the dual slack of the constraint wa, <c, is basic. Assuming dual nondegener- 
acy, this dual slack variable must be positive so that wa, <c,, that is, z, —c, < 


0. As discussed, this would guarantee finite convergence, since the dual 
objective strictly increases at each iteration. 

In the presence of dual degeneracy, however, we need a cycling preven- 
tion rule in order to guarantee finite convergence. From the discussion in 
Section 4.6 and noting that the dual simplex method is essentially applying the 
ordinary simplex algorithm to the dual problem, we can readily devise cycling 
prevention rules. In particular, the following lexicographic cycling prevention 
rule may be used. Suppose that we have a dual feasible tableau for which 


ib a is lexicographically positive for each nonbasic variable column. Note 
J 


that this is the updated nonbasic variable column, with the negative of the 
objective row coefficient. A technique to obtain a starting tableau satisfying this 
property is presented in Section 6.6. Then, having selected a pivot row r in the 


dual simplex tableau with b, < 0, we can scale each of these lexicographically 


positive nonbasic columns that have a negative Yj coefficient in row r by 
dividing it by —Yrj and picking the resulting lexicographically minimum column 


from among these as a pivot column. We ask the reader in Exercise 6.41 to 
show that this will maintain the lexicographic positivity property and, in fact, 
prevent cycling. Similarly, noting that every entering and leaving basic variable 
in the primal problem has a corresponding leaving and entering complementary 
variable in the complementary dual basis, Bland’s Rule may be readily adapted 
to prevent cycling in the dual simplex method. Identical to the discussion in 
Section 4.6, of all the candidates for pivot rows, and similarly for pivot columns, 
we can break ties (if any) by favoring the x;~—variable having the smallest 


index. We defer the details of this rule to Exercise 6.41. 


6.5 THE PRIMAL-DUAL METHOD 


Recall that in the dual simplex method, we begin with a basic (not necessarily 
feasible) solution to the primal problem and a complementary basic feasible 
solution to the dual problem. The dual simplex method proceeds by pivoting 
through a series of dual basic feasible solutions until the associated comple- 
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mentary primal basic solution is feasible, thus satisfying all of the KKT 
optimality conditions. 

In this section we describe a method, called the primal—dual algorithm, 
which is similar to the dual simplex method in that it begins with dual feasibility 
and proceeds to obtain primal feasibility while maintaining complementary 
slackness. However, an important difference between the dual simplex method 
and the primal—dual method is that the primal—dual algorithm does not require 
the dual feasible solution to be basic. Given a dual feasible solution, the primal 
variables that correspond to tight dual constraints (so that complementary slack- 
ness is satisfied) are determined. Using Phase I of the simplex method, we 
attempt to attain primal feasibility using only these variables. If we are unable to 
obtain primal feasibility, we change the dual feasible solution in such a way as 
to admit at least one new variable to the Phase I problem. This is continued until 
either the primal becomes feasible or the dual becomes unbounded. 


Development of the Primal—Dual Method 


Consider the following primal and dual problems in standard form where b > 0. 


P: Minimize cx D: Maximize wb 
subject to Ax = subject to wA < ¢ 
x > 0. Ww unrestricted. 


Let w be an initial dual feasible solution, that is, wa; < c; for all j. By comple- 


mentary slackness, if wa pep then x; is allowed to be positive and we 


attempt to attain primal feasibility from among these variables. Let 0 = {: wa, 


— c; = O}, that is, Q is the set of indices of primal variables allowed to be 


positive. Then the Phase I problem that attempts to find a feasible solution to the 
primal problem among variables in the set O becomes: 


Minimize > Ox; + Ix, 


jeQ 
subjectto DY aj;x; + X, = b 
JeQ 
xj, 2 0 for j€Q 
= 0. 


a 


We utilize the artificial vector x, to obtain a starting basic feasible solution to 
the Phase I problem. The Phase I problem is sometimes called the restricted 
primal problem. 

Denote the optimal objective value of the foregoing problem by xg. At 
optimality of the Phase I problem either x9 = 0 or xg > 0. When xq = 0, we have 
a feasible solution to the primal problem because all the artificial variables are 
zeros. Furthermore, we have a dual feasible solution, and the complementary 
slackness condition (wa; — c;)x; = 0 holds true because either j € Q, in which 


case wa; —c; = 0, or else 7 ¢ Q, in which case xy = 0. Therefore, we have an 
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optimal solution of the overall problem whenever x9 = 0. If x9 > 0, primal 
feasibility is not achieved and we must construct a new dual solution that admits 
a new variable to the restricted primal problem in such a way that xg might be 
decreased. We shall modify the dual vector w such that all the basic primal 
variables in the restricted problem remain in the new restricted primal problem, 
and in addition, at least one primal variable that did not belong to the set Q is 
passed to the restricted primal problem. Furthermore, this variable is such that it 
reduces xg if introduced in the basis. In order to construct such a dual vector, 


consider the following dual of the Phase I problem: 


Maximize vb 


subjectto va; < 0, jeQ 
Vv < 1 
Vv unrestricted. 


Let v" be an optimal solution to the foregoing problem. Then, if an 


original variable x; is a member of the optimal basis for the restricted primal, 


the associated dual constraint must be tight, that is, v'a fo 0. Also, the criterion 
for basis entry in the restricted primal problem is that the associated dual 
constraint be violated, that is, va or 0. However, no variable currently in the 
restricted primal has this property, since the restricted primal has been solved 
optimally. For j ¢Q, compute v'aj;. If va; > 0, then if x; could be passed 
to the restricted primal problem it would be a candidate to enter the basis with 
the potential of a further decrease in x9. Therefore, we must find a way to force 
some variable xj with va; > 0 into the set Q. 


Construct the following dual vector w’, where @> 0: 
WwW =w+Ov'. 
Then, 


w'a.-c, = (w+Ov')a.-c; 
( Maj ee (6.12) 
7 (wa; —c;)+A(v a ;). 


Note that wa; —c; = 0 and va; <0 for 7€Q. Thus, Equation (6.12) implies 


that w’a;—c; <0 for j¢Q. In particular, if x; with j <¢Q isa basic variable 


in the restricted primal, then v"a ; = 0 and w‘a; — c; = 0, permitting j in the new 


J j 
restricted primal problem. If 7 #Q and v'a j <9, then from Equation (6.12) 


and noting that wa; —c; <0, we have w'a; —c; <0. Finally, consider j ¢ Q 


with v'a 7 7 0. Examining Equation (6.12) and noting that wa prep = 0 for 
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--¢.< 
; c; <0 for 


J #Q with at least one component equal to zero. In particular, define @ as 


j €Q, it is evident that we can choose a 0 > 0 such that w'a 


follows: 


—(wa, — _ —(wa,—c;) 
O= ass) = minimum a 
Vv a yi va; 

By the definition of @ and from Equation (6.12), we see that w'a, —c, = 0. 


:v'a, >0>>0. (6.13) 


Furthermore, for each j with v‘a ; 7 9, and noting Equations (6.12) and (6.13), 


we have w'a; See 0. 


To summarize, modifying the dual vector as detailed previously leads to a 


new feasible dual solution where w’a pes < 0 for all 7. Furthermore, all the 


variables that belonged to the restricted primal basis are passed to the new 
restricted primal. In addition, a new variable x, that is a candidate to enter the 


basis, is passed to the restricted primal problem. Hence, we continue from the 
present restricted primal basis by entering x,, which leads to a potential reduc- 


tion in Xp. 
Case of Unbounded Dual 


The foregoing process is continued until either xg = 0, in which case we have 


an optimal solution, or else, x9 > 0 and v'a pe 0 for all 7 @Q. In this case, 


consider w= w+Ov". Since wa, —c, <0 for all j and by assumption va; <0 


for all j, then from Equation (6.12) w’ is a dual feasible solution for all 6 > 0. 
Furthermore, the dual objective value is 


w'b = (w+ Ov" )b = wb+ Ov b. 


Since v"b = x9 (why?) and the latter is positive, then w’b can be increased 


indefinitely by choosing @ arbitrarily large. Therefore the dual is unbounded and 
hence, the primal is infeasible. Alternatively, note that the fofal (usual) Phase I 
problem has the current restricted Phase I solution as an optimum (why?). Since 
Xg > 0, the primal is infeasible. 


Summary of the Primal-Dual Algorithm (Minimization Problem) 


INITIALIZATION STEP 


Choose a vector w such that wa; —c; <0 for all /. 
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MAIN STEP 
1. Let O= {j: wa; —c; = 0} and solve the following restricted primal 
problem: 
Minimize > Ox; + 1x, 
jeQ 
subjectto Yo aj;x; + Xqg = b 
jeQ 


x; 2 0 forjeQ 

x, 2 0. 
Denote the optimal objective value by xp. If x9 = 0, stop; an optimal 
solution is obtained. Otherwise, let v" be the optimal dual solution 
to the foregoing restricted primal problem. 

2. If va; <0 for all 7, then stop; the dual is unbounded and the primal 


is infeasible. Otherwise, let 


oe, —(wa ;—C;) 
6 = minimum, ——2_~ 


J va; 


:v'a; >0 > 0, 


and replace w by w+ Ov". Repeat Step 1. 


Example 6.8 


Consider the following problem: 
Minimize 3x, + 4x. + 6x3 + 7x4 + = X5 


subjectto 2x, -— X27 + X3 + 6x4 — 5x5 - X = 6 
Hot Xp + 2x, + Xqg + 2X5 — x7 = 3 
X15 xX, x3, X45 X55 X6> xX, 2 0. 


The dual problem is given by the following: 


Maximize 6w, + 3wy 
subjectto 2w, + w, < 3 
6w, + W) <7 
Sw a 2w, < 1 
W2 < 0 
Ww, w, unrestricted. 


An initial dual feasible solution is given by w = (w,,w2) = (0,0). Substituting w 
in each dual constraint, we find that the last two dual constraints are tight so that 
QO = {6, 7}. Denoting the artificial variables by xg and x9, the restricted primal 
problem becomes as follows: 
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Minimize xg + Xp 
subject to —x¢ + Xx 
— X47 + X9 


6 
3 
X65 x7, Xg, Xo 0. 


Mi ll 


The optimal solution to this restricted primal is clearly (x¢,x7,xg,X9) = (0,0,6,3) 
and the optimal objective value x) = 9. The dual of the foregoing restricted 
primal is the following: 


Maximize 6y, + 3v) 


subject to —y < 0 
— V2 < 0 
Vy) < 1 
v2 < 1 
V5 Vv) unrestricted. 


Utilizing complementary slackness, we see that since xg and Xo are basic, the 
last two dual constraints must be tight and v" = (v;, v3) = (1, 1). Computing 
va, for each column j, we have v’a; =3, va, =0, v'a3 =3, v'ay =7, and 
v’a; = —3. Then Gis determined as follows: 

@ = minimum {-(—3/3), -(— 6/3), -(— 7/7)} = 1, 


and w’ =(0,0)+ 1(1, 1)=(1, 1). 
With the new dual solution w’ we recompute Q and obtain Q = {1, 4}, 
giving the following restricted primal: 


Minimize xg + Xp 


subject to 2x, + 6x4 + Xx = 6 
Xx + X4 + Xyg = 3 
x}, X4, xg, X9 > 0. 


This time an optimal solution to the restricted problem is given by 
(x 3X45%8,X9) = (3,0, 0, 0), 
with x9 = 0. Thus we have an optimal solution to the original problem with the 


optimal primal and dual solutions being 


(x7 9 ,X3,X45X5,X6X7) = (3,0,0,0, 0, 0,0), and (ww) =(1, DD. 


Tableau Form of the Primal—Dual Method 

Let z pees be the row zero coefficients for the original primal problem, and let 
Zz j -¢ '; be the row zero coefficients for the restricted primal problem. Then for 
each original variable x;, we have 


Zp 7c; =Wa;—c; and z,—€; = va; —0= va;. 
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We also have 


and 

(wa, —c;)+ Ova; =(z; —c;)+ OZ; =Ga)s 
We can carry out all of the necessary operations directly in one tableau. In this 
tableau we have two objective rows; the first gives the (z fis )-values, and the 


second gives the (2; —¢,)—values. We shall apply this tableau method to the 


foregoing problem. The initial tableau is displayed below. In this example, w is 


initially (0, 0), so that z;-c; =wa;-—c,; =—c, and the right-hand-side entry 


in the z—-row is zero. When w # 0, we still compute zj—€j = Wa; —C;, but also 


J 5 ee) 
initialize the RHS entry of the z—-row to wh instead of zero. [Try initializing the 
tableau with w = (1, 0).] 


X} Xp. %, Xqy Xs Xp X7 Xg Xq RHS 


Since we begin with xg and x in the basis for the restricted primal, we must 


perform some preliminary pivoting to zero their cost coefficients in the Phase I 
objective. We do this by adding the first and second constraint rows to the 


restricted primal objective row. Then 2; —¢,; = 0 for the two basic variables xg 


and xg. Let c indicate the variables in the restricted primal problem, that is, 
those for which z;—c; = 0. As the restricted primal problem is solved, only the 


variables flagged with c are allowed to enter the basis. 


U U U 
x X2 x3 x4 X5 X6 x7 Xg Xg RHS 
Zz 
x0 
8 
9 


Since Z;—¢; <0 for all variables in the restricted problem, we have an optimal 


solution for Phase I. Then @is given by 
Ve —(z;—¢;) j 
O= niin LED, —-é,>0 


zp oj 


= minimum {-(- 3/3),-(— 6/3), -(- 7/71)} =I. 


292 Chapter 6 


Thus we add one times the Phase I objective row to the original objective row. 
This leads to the following tableau. The Phase I problem is solved by only 


utilizing the variables in the set Q, that is, those with z; —c; = 0. 


Zz 

*0 

Xg 

x9 

Zz = a 

Xp sie 11/6 ie 6-1 

X4 -1/6 1/6 -5/6 -1/6 : 

Xp D 7/6 11/6 17/6 1/6 

Zz 

Xo 

x4 = a7 a a ii 7 
x 11/4 17/4 1/4 -6/4 -1/4 6/4 


Because xq = 0, the optimal solution is found, namely: 


(x1 5 X2,X3,%45.X5,%6,X7) = (3,0,0,0,0,0, 0) 


whose objective value is 9. 


Finite Convergence of the Primal—Dual Method and an Insight 


Recall that at each iteration an improving variable is added to the restricted 
primal problem. Therefore, in the absence of degeneracy in the restricted primal 
problem, the optimal objective x9 strictly decreases at each iteration. This 
means that the set O generated at any iteration is distinct from all those gener- 
ated at previous iterations. Since there is only a finite number of sets of the form 
QO (recall Oc {1,2,...,}) and none of them can be repeated, the algorithm ter- 
minates in a finite number of steps. 

Let us provide an insight into the primaldual algorithm that readily 
resolves the convergence issue under degeneracy, following Section 4.6. Note 
that the primal—dual method is essentially working on the otal Phase I problem 
of minimizing the sum of the artificial variables 1x, subject to Ax + x, =b, x 


> 0 and x, > 0. However, it is restricting the nonbasic variables that can be 
chosen to enter the basis via the dual vector w. At any tableau, either we have 
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2;—¢; = va;< 0 for all j, in which case the total Phase I problem has been 


solved, or we have Z - -¢ 7 = Va; > 0 for some nonbasic variables. In the former 


case, if xq > 0, then because the (total) Phase I problem has a positive optimal 
solution value, the primal is infeasible. On the other hand, if x) = 0, then the 


current primal—dual pair (x, w) satisfies the KKT conditions, and is hence 
primal—dual optimal. Otherwise, if the total Phase I problem has not been solved 
as yet, the manipulation of w guarantees that at each iteration, some nonbasic 


variable x; having 2; —¢; = va; > 0 is introduced into the set Q, and is hence 


enterable. Therefore, we are essentially using the primal simplex algorithm on 
the (total) Phase I problem along with a restricted entering variable criterion. 
Consequently, by employing the usual lexicographic rule of Section 4.6, which 
is independent of how one selects an entering variable, we can obtain finite 
convergence even in the presence of degeneracy. The minor details of this 
implementation are left to the reader in Exercise 6.50. 


6.6 FINDING AN INITIAL DUAL FEASIBLE SOLUTION: 
THE ARTIFICIAL CONSTRAINT TECHNIQUE 


Both the dual simplex method and the primal—dual method require an initial 
dual feasible solution. In the primal tableau, this requirement of dual feasibility 
translates to z; —c; <0 for all for a minimization problem. We shall now see 


that this can be accommodated by adding a single new primal constraint. 
Suppose that the first m columns constitute the initial basis and consider 


adding the constraint > <M, where M> 0 is large. The initial tableau 


n 
j=m+l xj 


is displayed below, where x,,,, is the slack variable of the additional constraint. 


This additional constraint bounds the nonbasic variables, and thus indirectly 
bounds the basic variables and thereby the overall primal problem. To obtain a 
dual feasible solution in the new tableau we let 


Zh — Ck sean {z;—c;}. 


Once column k has been selected, we perform a single pivot with column & as an 
entry column and column n + | as an exit column. In particular, to zero z, —c, 


we shall subtract z,—c, times the new row from the objective function row. 
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Note that the choice of & and the single pivot described ensure that all new 
entries in row 0 are nonpositive, and thus we have a (basic) feasible dual 
solution. With this starting solution, either the dual simplex method or the 
primal—dual simplex method can be applied, eventually leading to one of the 
following three cases: 


1. The dual problem is unbounded. 
2. Optimal primal and dual solutions are obtained with x), > 0. 


3. Optimal primal and dual solutions are obtained with x,,, =0. 


In Case | the primal problem is infeasible. In Case 2 we have an optimal 
solution to the primal and dual problems. In Case 3, the new bounding 
constraint is tight at optimality. If z,.; —Cc,4; <0, then x,,; is nonbasic and 
this new bounding constraint limits the primal solution. As M increases, the 
objective will continue to reduce indefinitely; hence, the primal problem is 
unbounded. However, if 2,1; —Cy4; = 0, then the current primal solution is 
optimal (though not basic to the original problem). In Exercise 6.43 we ask the 
reader to explore this case in detail. 

Furthermore, in Exercise 6.44 we ask the reader to show that applying the 
artificial constraint technique to the primal problem is equivalent to applying the 
single artificial variable technique (described in Chapter 4) with the big- 
method to the dual problem and vice versa. 


Example 6.9 


Suppose that we wish to apply the dual simplex method to the following tableau: 


Zz x Xo X3 X4 X5 RHS 


Adding the artificial constraint x. + x3 + x4 < Mwhose slack is xg, we get 
the following tableau: 


From this tableau we find that maximum {z; —c;} = 23 —c3 = 5. Pivoting in the 


x3 column and the x¢ row, we get the following new tableau that is dual 
feasible. The dual simplex method can now be applied in the usual manner. 
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6.7 SENSITIVITY ANALYSIS 


In most practical applications, problem data are not known exactly and are often 
estimated as best as possible. It is therefore important to study the effect on 
optimal solutions for the problem to variations in certain data, without having to 
resolve the problem from scratch for each run. Also, at early stages of problem 
formulation, some factors might be overlooked from the viewpoint of analytical 
simplicity. It is useful to explore the effect on the current solution of accommo- 
dating some of these factors. Furthermore, in many situations, the constraints are 
not very rigid. For example, a constraint might reflect the availability of some 
resource. This availability can be increased by extra purchase, overtime, buying 
new equipment, and the like. It is desirable to examine the effect of relaxing 
some of the constraints on the value of the optimal objective value without 
having to resolve the problem. Moreover, the principal utility of a model is not 
simply to determine an optimal solution or policy for a given problem or 
situation, but rather to provide a facility (or an oracle) to derive quantitative 
insights into the modeled system by posing various “what-if’ queries such as: 
what might be the effect of alterations in certain key influential exogenous or 
endogenous parameters on the optimal solution; or what might be the benefit of 
investing in some new potential option or activity or technology; or how would 
the system be perturbed if we shut down an ongoing operation? The 
investigation of these and other related issues constitutes sensitivity analysis or 
what-if analysis. In this section, we shall discuss some methods for updating an 
optimal solution under different problem variations. 
Consider the following problem: 


Minimize ex 
subject to Ax b 
x 


> 0. 


Suppose that the simplex method produces an optimal basis B. We shall 
describe how to make use of the optimality conditions (primal—dual relation- 
ships) in order to find a new optimal solution, if some of the problem data 
change, without resolving the problem from scratch. In particular, the following 
variations in the problem will be considered. In doing so, we will also provide 
some further insights into duality. 


Change in the cost vector ec. 

Change in the right—hand-side vector b. 
Change in the constraint matrix A. 
Addition of a new activity. 
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Addition of a new constraint. 


Change in the Cost Vector 
Given an optimal basic feasible solution, suppose that the cost coefficient of one 
(or more) of the variables is changed from c, to cy. The effect of this change 


on the final tableau will occur in the cost row; that is, dual feasibility might be 
lost. Consider the following two cases: 


CaselI: x, Is Nonbasic 


In this case, ¢p is not affected, and hence, z j =e zB 'a j is not changed for 
any j. Thus, z, —cg is replaced by z, —c,. Note that zy, —cy < 0, since the 
current basis was optimal to the original problem. If z,-—c, = (zR -—cx) + 
(cy —Ccj ) 18 positive, then x, must be introduced into the basis and the (primal) 


simplex method is continued as usual. Otherwise, the old solution is still optimal 
with respect to the new problem. 


Case II: x, Is Basic, Say, x, = xg, 


’. Then z'-—c, is 


Here, CR, is replaced by CR. Let the new value of z j be z; ang, 


calculated as follows: 
f t -_ -! t 
2, = CRB 'a;—c; =(CgB a; cj) +(0,0,....CR, — Cp, .On---s OY ; 


=(z; —cj)+ (cp ~Cg )¥y for all. 
In particular, for j =k, z, -—cy =0, and yg = 1, and hence, z; —cy =c, —Cx. 
As we should expect, z, —c; is still equal to zero. Therefore, the cost row can 
be updated by adding the net change in the cost of Xp = xX, times the current ¢ 
row of the final tableau, to the original cost row. Then, z, —c, is updated to 
Zz, —C, = 0. Of course the new objective value c,B™'b = zB 'b + 


(cz, -C, )b, will be obtained in the process. 


Example 6.10 
Consider the following problem: 
Minimize -2x, + Xx, - 3 
subjectto x + X% + % S 6 
—Xy + 2x5 <4 
X45 X45 X3 2 0. 


The optimal tableau is given as follows: 
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Suppose that cy = 1 is replaced by —3. Since x, is nonbasic, then z)-c, = 


(22 —€2) + (€2 -¢)) = -3 + 4 = I, and all other z; —c; are unaffected. Hence, 


X, enters the basis in the following tableau: 


Z XxX 3 XX,  X5  RHS 


The subsequent tableaux are not shown. Next, suppose that c; = -—2 is replaced 
by zero. Since x, is basic, then the new cost row, except z, —c,, is obtained by 
multiplying the row of x, by the net change in c, [that is, 0 — (-2) = 2] and 
adding to the old cost row. The new z,;—c,; remains zero. Note that the new 
23 —C3 iS now positive and so x3 enters the basis in the following tableau: 


The subsequent tableaux are not shown. 
Change in the Right-Hand-Side 


If the right-hand-side vector b is replaced by b’, then B™'b will be replaced 
by B-'b’. The new right-hand-side can be calculated without explicitly 
evaluating B~'b’. This is evident by noting that B-'b’ = B-'b + B|(b’ — b). If 


the first mm columns originally form the identity, then B|(b’ - b) = 


Lay Oj —5;), and hence B™ b= b+ age 1y (Bj - —b,). Since z;-c; <0 
for all nonbasic variables (for a minimization problem), the only possible 
violation of optimality is that the new vector Bb’ may have some negative 
entries. If B-'b’ > 0, then the same basis remains optimal, and the values of the 
basic variables are B-'b’ and the objective has value cgB™'b’. Otherwise, the 


dual simplex method can be used to find a new optimal solution by restoring 
primal feasibility. 
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Note that this case is similar to the previous case in that we are changing 


the cost vector in the dual problem. Hence, if Bb’ > 0, then dual optimality is 
maintained. Otherwise, in the available dual basic feasible solution, the nonbasic 
dual variable complementary to a negative basic primal variable is enterable, 
and we proceed by applying the simplex algorithm to the dual as previously. 


Example 6.11 


Suppose that the right-hand-side of Example 6.10 is replaced by (3). Note that 


B! |i Ht and hence Bb’ =|1 ilal=(). Then, Bb’ > 0, and 


hence the new optimal solution is x, =3, x5 =7, X27 = x3 = x4 =0. 


Change in the Constraint Matrix 


We now discuss the effect of changing some of the entries of the constraint 
matrix A. Two cases, namely, changes involving nonbasic columns and changes 
involving basic columns are discussed. The dual operations of changing rows in 
A are also discussed. 


Case I: Changes in Activity Vectors for Nonbasic Columns (Changes in 
Rows with Basic Slack Variables) 


, is modified to a’;. Then, the new updated 


Suppose that the nonbasic column a ; j 


. -lo ' = on i co A. . 
column is B’‘a’; and z; —c; = ¢gB a’; ~ c;. If 2; —c; < 0, then the old solu 


tion is optimal; otherwise, the simplex method is continued, after column / of 
the tableau is updated, by introducing the nonbasic variable x ;. 


Changing a nonbasic variable column in the primal corresponds to chang- 
ing a dual constraint for which the slack variable is basic. Hence, the dual 
operation for this case identifies with changing the row (in a “primal” problem) 
where the associated slack variable is basic. In this case, the new row simply 
replaces the old row and is then updated to bring the tableau into canonical 
form. If the slack variable is negative when this is accomplished, then we may 
continue by using the dual simplex algorithm, starting by pivoting on this row, 
and hence, entering the complementary dual variable into the basis. 


Case II: Changes in Activity Vectors for Basic Columns (Changes in Rows 
with Nonbasic Slack Variables) 


Suppose that a basic column a; is modified to a’;. Now, it is possible that the 


current set of basic vectors no longer form a basis after the change. Even if this 
does not occur, a change in the activity vector for a single basic column will 


change p | and thus the entries in every column. 
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This change is handled very easily in two steps. First, assume that a new 
activity xy is being added to the problem with column a’; (and objective coef- 


ficient c;). Second, eliminate the old variable x; from the problem. The first 
step is accomplished by computing y’; = Bla’, and z’;—c; = ¢gB™ al; — c; 
where B is the current basis matrix. This gives the updated column for x’. If the 


element yj; in the basic row for x; and the column of x; is not zero, then x} can 


be exchanged for x; in the basis and the column of the old variable x; can be 


eliminated from the problem (why?). This pivot might destroy either or both 
primal and dual feasibility, but we can restore primal or dual feasibility using 
artificial variables if necessary and reoptimize. On the other hand, if y‘; = 0, 


then the current set of basic vectors with the new column for x j no longer forms 


a basis (why?). In this case, one way to eliminate x; from the problem is to treat 


it as an artificial variable that is basic in that row and to resort to the two-phase 
or the big—M method. 

The dual operation for this case is to change a row in the problem where 
the corresponding slack variable is nonbasic. Since several coefficients in the 
updated slack variable column can be nonzero, this row may have multiples of it 
added to several other rows. For convenience, let us refer to the slacks in the old 
and the new form of this ith constraint as x,; and x{;, respectively. Then the 
dual operations to the ones given previously are as follows: First, add this new 
row with x,; basic and update the tableau into canonical form. (Some related 
mathematical details are given later in the discussion on adding a new 
constraint.) If the element in this row and the column of x,; is nonzero, then 


exchange x,; for x,; in the basis. (This corresponds to the case y'; # 0 above.) 


Delete the representation of x,; in terms of the nonbasic variables from the 
problem, since the old form of the constraint is no longer needed. Otherwise, if 
the exchange of x{,; for x,; in the basis is not possible, we can change the right— 
hand-side of the old (< type) constraint to a big—M value in order to make this 
old form of the constraint essentially redundant to the problem and continue the 
optimization by using the dual simplex method. This corresponds to the 
handling of the case yj; = 0. 


Example 6.12(a) 


2 


Suppose that in Example 6.10, a, is changed from (3) to (3 


yo=Ba-(1 9}(3}=(3) 


) Then, 
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ope lao (-2,0)(3}-1 7 


Thus, the current optimal tableau remains optimal with column x, replaced by 


(-5, 2,7) : 


Next, suppose that column a, is changed from (3) to (_{). Then, 


yieeta=(! %)(%)=(.9) 


cB 'a' —c¢, = (-2,0)(_{)--2)= oe 
Here, the entry in the x, row of yj is zero, and so the current basic columns no 


longer span the space. Adding the column (2,0,-1)' of xj and making x, an 


artificial variable in the basis with a big—M penalty in the objective function, we 
get the following tableau: 


After preliminary pivoting at row x, and column x, to get z, —c, = 0, that is, 
to get the tableau in canonical form, we can proceed with the big—M/ method. 


Finally, suppose that column a, is changed from is) to a Then, 


i}  \6 
iret 6 
cpB"la’ —c, = (-2,0)(3)-(-2) ad: 


In this case, the entry in the x, row of y; is nonzero and so we add the column 


(-4,3,9)' of xj, perform a pivot in the x; column and the row for x, in the 
following tableau, and proceed with x, eliminated: 


, 


x X  xX3 X4 %\%  RHS 


z 
x ae aa 
1 1 1 


A 
43 
Gy) 1 
9 3 


The subsequent tableaux are not shown. 


x5 
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Example 6.12(b) 


Suppose that in Example 6.10 the first constraint is changed to x» — x3 <6. 
Call the new slack variable x4 and add this constraint to the optimal tableau 
with x, basic in the new row. This gives the following tableau (in canonical 
form): 


ZX  X. %, XX Xs x4  RHS 


Note that the element in row x4 and column xq is zero. (Otherwise, we could 
have pivoted on this element and deleted the resulting representation of x4 in 


terms of the nonbasic variables in the last row.) Hence, we change the right— 
hand-side of the old first (< type) constraint to M. This makes the updated 
right-hand-side vector equal to (-2M, M, M + 4, 6). Note that we still have 
optimality in this case, with the objective value of order —M. (The old constraint 
is restricting the optimum with z,-c, < 0.) Hence, the new problem is 


unbounded. 


Adding a New Activity 


Suppose that a new activity x,,; with per-unit cost c,,, and consumption 
column a,,, is considered for possible inclusion within the model. Without 
resolving the problem, we can easily determine whether involving x,4; is 


worthwhile. First, we calculate z,4; — Cyij- If Zpit — Cpa, < 0 (for a 


minimization problem), then x,,, = 0 and the current solution is optimal. On 


the other hand, if z,.; — ¢ 


n+] > 9, then we introduce x,,,, into the basis and 


use the simplex method to continue to find a new optimal solution. 
Example 6.13 


Consider Example 6.10. We wish to find a new optimal solution if a new 


activity x5 =O having cg = 1 and ag = ; is introduced. First, we calculate 
276 C6 and Ye: 


26-6 = Wag -c6 


It 
i 
N 
=) 
— 
ps 
ee 
NWS 
| 
Ul 
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Therefore, xg is introduced in the basis by pivoting at the xs row and the x¢ 
column in the following tableau: 


Z XM XX % XA %Xy; tr  RHS 
0 3 j-7| 
0 1 1 1 1 


0 0 3 1 1 


The subsequent tableaux are not shown. 


Adding a New Constraint 


Suppose that a new constraint is added to the problem. As discussed in the 
section on modifying the A matrix, this operation is dual to that of adding a new 
activity. If the given optimal solution to the original problem satisfies the added 
constraint, it is then obvious that this point is also an optimal solution of the new 
problem (why?). If, on the other hand, this point does not satisfy the new 
constraint, that is, if the constraint “cuts away” the given optimal solution, we 
can use the dual simplex method to find a new optimal solution. These two 
cases are illustrated in Figure 6.5, and they correspond to the two foregoing 
cases related to adding a new activity. 


Suppose that B is the optimal basis that is available before the constraint 
aly < b,,41 1s added. The corresponding canonical system is as follows: 


z+(¢gB 'N-cy)xy =¢gB 'b 


x; +B 'Nxy =B'b. (6.14) 


The constraint a”*!x < b,,,,, is rewritten as a*!x, +a%*!y  +,41 =P 4] where 


a”*! is decomposed into (a%*!, at!) and x,41 is a nonnegative slack variable. 


New constraint 
New optimum 


ae 


New constraint 


* 
x 


c 


Figure 6.5. Addition of a new constraint. 
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Multiplying Equation (6.14) by ag and subtracting from the new constraint 


gives the following system: 


z+(cgB 'N-cy)xy = ¢gB'b 
xp+B'Nxy = Bb 
(ant! ag BIN) xy + xp = Byatt Bb. 


These equations give us a basic solution of the new system (why?). The only 
possible violation of optimality of the new problem is the sign of 5,4) - 


a” Bp. So if B41 —a7*'B b> 0 , then the current solution is optimal. 


Otherwise, if 5,,.1 —a*lB lb <0, then the dual simplex method can be used 


to restore primal feasibility, and hence, restore optimality. 

Note that if an equality constraint is added after bringing the tableau to 
canonical form, we can use an artificial variable as a basic variable in this row 
and continue with the primal two-phase or the big—M method. The dual opera- 
tion is to add an unrestricted variable x j- The equivalent dual technique is to 


add the updated version of this column to the tableau. Since the variable is unre- 
stricted, we can use its positive or negative column and thus assume that the 
variable x 7 has z;-c; <0. Now, we impose the bound restriction x; => —M. 


j poo 
Using the transformation xj =x;+M20 to obtain a nonnegatively 
constrained variable xj in lieu of x;, we can continue optimizing with the dual 


simplex method. Alternatively, we could have used the transformation 


x; = % mies with x; xj >0, and added the two columns of xj and x} to the 


problem in lieu of that for x;. (What is the equivalent dual technique?) 


Example 6.14 


Consider Example 6.10 with the added restriction that —x, +2x; 22. Clearly, 
the optimal point (x),x2,x3) = (6,0,0) does not satisfy this constraint. The 
constraint —x, +2x3 22 is rewritten as x, —2x3+xg =—2, where x6 is a non- 


negative slack variable. This row is added to the optimal simplex tableau of 
Example 6.10 to obtain the following tableau: 
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Multiply row 1 by —1 and add to row 3 in order to restore column x, to a unit 


vector. The dual simplex method can then be applied to the following resulting 
tableau: 


XX Xp XMS a RHS 


Subsequent tableaux are not shown. 


An Application of Adding Constraints in Integer Programming 


The linear integer programming problem may be stated as follows: 


Minimize ex 
subject to Ax 
x 


b 
0. 
x integer. 


V Il 


A natural method to solve this problem is to ignore the last condition, x integer, 
and solve the problem as a linear program. At optimality, if all of the variables 
have integral values, then we have an optimal solution to the original integer 
program (why?). Otherwise, consider adding a new constraint to the linear pro- 
gram. This additional constraint should delete the current optimal noninteger 
linear programming solution without cutting off any feasible integer solution. 
Such a constraint is referred to as a valid inequality or a valid cut. Adding the 
new constraint to the optimal tableau, we apply the dual simplex to reoptimize 
the new linear program. The new solution is either integral or not. The 
procedure of adding constraints is repeated until either an all—integer solution is 
found or infeasibility results (indicating that no integer solution exists). How, 
then, can such a cutting plane be generated? 

Consider any optimal simplex tableau when a noninteger solution results. 


Let b, be nonintegral. Assume that the basic variables are indexed from 1 to m. 
The equation associated with b, is 
x, + 3 Vix; = =% 

j=m+} 
Let /,; be the greatest integer that is less than or equal to y,;(/,; is called the 
integer part of y,;). Similarly, let /, be the integer part of b,.. Let F,; and F, 
be the respective fractional parts, that is, 

Fy=Yy-lj; and F,=6,-1,. 

Then 0< F,; <1 and 0 < F, <1 (why?). Using this, we may rewrite the basic 


equation for x, as 
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nh 
Xt LY Cy +hy)x; =1,+F,- 


j=m+l 


Rearranging terms, we get 


nv n 
rt Do Lyx lp HF LF s;- 
j=m+l j=m+l 
Now, the left-hand-side of this equation will be integral for any feasible integer 
solution (why?). The right-hand-side is strictly less than 1, since F, <1, 


F,; 20, and x; 2 0. But since the right-hand-side must also be integral because 
it equals the left-hand-side, we may conclude that it must be less than or equal 
to zero (there are no integers greater than zero and less than one). Thus, we may 


write 


n 
F.- > Fx, <0. 


r qj 
j=m+l 


However, since x, is currently nonbasic (and hence x; =0) forj=m+1,...,7 


j 
and F.. > 0, the current optimal (noninteger) linear programming solution does 
not satisfy this additional constraint. In other words, this new constraint will cut 
off the current optimal solution if added to the current optimal tableau. The dual 
simplex method can then be applied to obtain a new optimal linear programming 
solution, which can be again tested for integrality. Such a procedure is called a 
cutting plane algorithm. This cut is called Gomory’s dual fractional cut. 


Example 6.15 


Consider the following integer program: 
Minimize 3x, + 4x 


subjectto 3x, + x, 2 4 
xX) + 2x, 2 4 
xX, x2 > 0 
Xy, x2 integer. 


In Figure 6.6 we show the optimal linear programming and integer program- 
ming solutions, respectively. 

Ignoring the integrality restrictions, the following tableau gives the opti- 
mal linear programming solution: 


Xx X4 RHS 


z Cotas Ta 
4 -2/5. +s | 4/5 
x 5-3/5 


Since this solution is nonintegral, we may select a fractional variable for 
generating a cut (including z). Select x,. (Note: Selecting different variables 


may generate different cuts.) The equation for the basic variable x, is 
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Feasible integer points 


Optimal 
linear 


: Optimal integer point 
point e eal 


xy 


Cut 


(1) 
Figure 6.6. A graphical solution for Example 6.15. 


Xq +1/5.x3 -3/5x4 =8/5. 

From this we get 
In, =0, Ing = 1, Ip =1, 
Fog = 5, Fog =2/5, Fy =3/5, 

and the additional constraint becomes 

1/5x3 +2/5x4 23/5 (cut). 
Adding this constraint with slack variable x5 to the tableau and applying the 
dual simplex method, we get the following tableaux: 


Z MN xX x3 X4  X5  RHS 
z O. vee aS 255 
x, 0 2/5 V/5 
X 1 5-3/5 
Xs 0 /S> -2/5 


Hence, we have obtained the optimal integer solution x* = (2,1)' with only one 
cut. If in the foregoing tableau some variable had turned out noninteger valued, 
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we would have generated a new cut and continued. In most integer programs, 
we might typically need to repeat this cutting plane process many times. 

It is interesting to examine the cut in terms of the original variables. 
Substituting x3 =3x,+x,—-4 and x4 =x,+2x)—4 into 1/5x3+2/5x4 23/5 
and simplifying, we get 

Xp +X, 23 (cut in terms of x; and x). 


It can easily be seen that the addition of this constraint to Figure 6.6 yields the 
required integer optimum. Finite convergence of such a procedure may be 
achieved by using an appropriate dual lexicographic pivot rule (see the Notes 
and References Section). 


Some Additional Issues in Sensitivity Analysis 


There are three pertinent issues related to sensitivity analysis in linear 
programming that need to be discussed. The first is performing sensitivity 
analysis in a revised simplex implementation. Here, the cases discussed 


previously are treated similarly, except that we only need to update ¢ Pe Bl, 


and Bb, as the case might be, in order to obtain the new revised simplex 
tableau. We hence leave the details to the reader in Exercise 6.55, but point out 


one important concept. Suppose that we have added a new row to the problem 


that has the coefficient vector ate in the columns of the current basic 


variables. Also, suppose that the slack or artificial variable x,,, whose column 
is a unit vector has been made basic in this new row. Then the new basis, Brey; 
and its inverse appear as given below: 


Xp Xnsl 
B 0 1 
Brew atl le Brew = 


This construct is useful in updating the revised simplex tableau in such a 
case. The second issue relates to performing sensitivity analysis for bounded 
variables linear programming problems. Again, the various cases discussed in 
this section (including changes in bounds on the variables) may be handled 
similarly, except that we need to ensure that the designated nonbasic variables 
are always at either their lower or upper bounds and that the basic variables are 
feasible to their bounds. Although the former condition is readily maintained, 
the following technique may be used in order to maintain the latter condition. 
Suppose that an update has been made while maintaining the nonbasic variables 
at their lower or upper bounds, but ignoring the bound restrictions on the basic 
variables. Let Ab be the vector having the smallest norm that needs to be added 
to the basic variable values in order to make them feasible to their bounds. Hence, 
Ab; = £ pj — Xp; if xpi < 2B; Ab; =Upi —Xp; If Xp; > Up, and Ab; =0 otherwise. 
Then, we introduce an artificial variable x,,, with —Ab as its updated column 
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and with 0<x,,, <1 as its bound restriction. Observe that by making x,,,, 


nonbasic at its upper bound we obtain a basic feasible solution to the artificial 
problem (why?). Hence, we may now proceed with the two-phase or the big-M 
method. (Alternatively, we can use a bounded variables version of the dual 
simplex method, if applicable (see Exercise 6.62).) We leave the details of the 
various Sensitivity analysis cases to the reader in Exercise 6.61. 

Finally, a third issue related to sensitivity analysis is the so—called toler- 
ance approach. Here, we are concerned with a tolerance on the simultaneous 
and independent variations in the coefficients of the objective function or of the 
right-hand—side so that the current optimal basis remains optimal. More 
precisely, consider the original and the perturbed linear programs P and P’, 


respectively, where c’., j = l,..., n, are given constants and a,, j = 1,..., 7, 
1 y yo J g j 


denote perturbation parameters, as follows: 


n 
P: Minimize Yc ;x 


ve 
subject to xa ayX 7 = 5; = fori=1,...,m 
x>0. 
P’: Minimize Le +a jC} )x; (6.15) 
subject to ¥ ayx; =8; fori =1,...,m 
j=l 
x20. 


Observe that if Cj =c,; forj=1.,..., 2, then 100a, gives a percentage variation 
(positive or negative) in the objective coefficient c,; of the variable x;. Given 
an optimal basis B for P, the question raised in this context is: What is the maxi- 


mum value a 20 such that whenever -a < a; < ag for each j = 1,..., m, we 


will still have B as an optimal basis in P’. The value ag is then a maximum 
allowable tolerance on variations in the objective coefficients with respect to c’. 


Note that when c’ = ¢, a gives a (maximum) permissible accuracy tolerance 
on estimating the (nonzero) objective coefficients so that current basis remains 
optimal. 

A symmetric question can be addressed with respect to right—hand—side 
perturbations using the original and perturbed linear programs P and P’, 
respectively, defined as follows: 
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n 
P: Minimize > c;x; 
j=l 


n 
subject to LX agx; = 8; fori=1,...,.m 
j=l 


P’: Minimize Dex} (6.16) 


n 
subjectto Y ajx; =); + Bjb; fori=l,...,m 
j=l 
x20. 


Here, we are interested in determining a maximum allowable tolerance By = 0 
on variations in the right-hand-side values with respect to b’ , given an optimal 
basis B for P such that whenever —{9 < f; < fo for each i = 1,..., m, we will 
still have B as an optimal basis in P’. 

In order to determine aj, let B denote an optimal basis for P, let cg;, 
Cp;, and @p; denote the corresponding basic variable coefficients in the vectors 
c, c’, and a, respectively, let J #@ denote the index set for the nonbasic 


variables, let y F =B la j denote the updated column for x;, jeJ, with 


coefficients yy, i= 1, ..., m, corresponding to the rows of the basic variables 


— _ =I : 
Xpi, i= 1, ..., m, and as usual, let zjy-€ =cpB ay—C; for jEJ. Then, B 


Jj 
remains an optimal basis in P’ if and only if the objective row coefficients in 
P’ are nonpositive for the canonical representation with respect to the basis B. 


In other words, we must have 


m 
X Cp; + pie) ¥y — (Cj +Ajcj) $0 foreach je J, 
i=l 


that is, 


m 
d Cp ¥y@pi — jC; S (Cj -2;) for each j € J. (6.17) 
i=] 


We now need to obtain @@ as the largest nonnegative value such that so long as 
ao <a,;< a, the value of the left-hand-side in Equation (6.17) is no more 


than (c;—z,;)20, foreach j¢J. But the largest value that the left-hand-side 


in Equation (6.17) can ever get subject to —ag < a; $a, j = 1,..., n, for any 


a 20 is (LM ilebi Vis +c’ ao. Therefore, we have 
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m 
a = maximus ag =0,( leBivy esl }eo S(c;-z,;) foreach j <1}. 
i=] 


(6.18) 
Denoting J* = {je J: 5 |ep:;;|+\c',] > 0}, we get in closed—form: 
J i=) |°Bi y J 


‘ Cj a Z; 
Q = minimum ———_——. (6.19) 


i + m 
fei" Zlebivy| +e) 
i=l 
Similarly, denoting the components of B! as By a i,j = 1,..., m, and 


letting b=B™'b, we have that B is an optimal basis in P’ of Equation (6.16) if 
and only if it is feasible, that is, aa (B;'(b; + Bb -)20 for each i = 1,..., 


This can be rewritten as requiring that 


m = 
Y (-By'b)B; <b; foreach i =,...,m. (6.20) 
j=l 


Given any fp 20, the largest possible value of the left-hand-side in Equation 
(6.20) subject to ~Ay < B; < By forj=1,.., m, is (I 1|35'6) Ao (why?). 


Hence, we obtain 
Bo = nxn | fy [s 8 ; |) <b, fori=l,. wh (6.21) 


Denoting J* = {i {l,...,m}: Bie A 'b;| > 0}, we get 


ys 
P bj 
Bo = ee Bip (6.22) 
iel ’ 
2 | By 5; 


j=l 

In closing this section, we remark that the foregoing analysis is only one 

type of tolerance issue that may be addressed. For example, we might wish to 
determine such maximum tolerances subject to other imposed restrictions on 
variations in objective or right-hand-side parameters. In other words, certain 
bounds on some of these parameters or certain interrelationships in their 
variations may be known a priori in terms of additional restrictions on the @ ;— 


or the #,-values. Such cases are readily analyzed by including these constraints 
along with —ap Sa; <a, j = 1,...n (or —B <B; <fo,J = 1,..., m) when 


determining the maximum possible value of the left-hand-side in Equation 
(6.17) [or in Equation (6.20)] before formulating and solving Equation (6.18) for 


a [or Equation (6.21) for fo ]. The following example illustrates this point: 
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Example 6.16 


Consider Example 6.10 with the optimal basis B = (a,,a5) as given there. 
Suppose that c’ = c. Hence, J = {2, 3, 4}, cp = (-2, 0), and we obtain 


{(c;-2;) FJ} = 3, 1,23, fej], Fe I} = C1 1, Of, and (OM legiyy |, Je J} 


= {2, 2, 2}. Hence, from Equation (6.19), we get a = minimum {3/3, 1/3, 2/2} 
1/3. Consequently, the objective coefficients can vary within a tolerance of 


MI 


33 ; percent and the given basis will remain optimal. 


Now, suppose that the objective coefficients of x), x4, and x5 are known 
with certainty to be —2, 0, and 0, respectively, but variations in c, and c3 are 
permissible. Then c’ = (0, 1, —1, 0, 0), and so AR jeJ} = (1, 1, 0} and ep 
= (0, 0). This gives J* = {2, 3}, and we get from Equation (6.19) that a = 
minimum {3/1, 1/1} = 1. Therefore, subject to the stated restrictions, a 100 
percent tolerance in variation is permissible. 


On the other hand, suppose that ec’ = c, but we know that the objective 
coefficient for x, will vary from its given value by no more than +25 percent. 


Now, the relationships (6.17) require that —2a@,-—a@) <3, —2a,+a3 <1, and 
—2a, <2. Hence, given that -ay <@; <@ for j = 1,...,5 and that -1/4 < ay < 


1/4, the maximum values of the left-hand-sides of the foregoing three inequali- 
ties are, respectively, 2[min{a,1/4}]+ ao, 2[min{ay,1/4}]+a@p, and 2[min{aq, 


1/4}]. We need to find the largest value a" of @ such that these quantities 
are respectively no more than 3, 1, and 2. Because we know that aj" > 1/3 
(why?), we have min{ag’, 1/4} = 1/4, and so, ay’ = max {a : 1/2+a $3, 


1/2+@p <1, and 1/2 < 2}. This gives ao = minimum {5/2, 1/2} = 1/2. Therefore, 
a 50 percent, instead of only a 33 percent, variation in the objective coeffi- 


cients is permissible, given a maximum variation of 25 percent in cy. 
Finally, let us illustrate the case of variations in the right—hand-side. 


Assuming that b’ = b, we have b = (6,10), B= i |. and 2m |B 'b)| equals 
6 and 10 for the first and second basic variable rows, respectively. Hence, from 


Equation (6.22), Bo = minimum {6/6, 10/10} = 1, that is, a 100 percent varia- 
tion in the right~hand-sides is permissible for the given basis to remain optimal. 
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6.8 PARAMETRIC ANALYSIS 


Parametric analysis is used quite often in large-scale linear programming and in 
nonlinear optimization, where we often find a direction along which the objec- 
tive function gradient or the right-hand-side vector of the constraints are 
perturbed, and then we seek to ascertain the resulting trajectory of optimal 
solutions. Hence, we are interested in determining optimal solutions to a class of 
problems by perturbing either the objective vector or the right-hand—side vector 
along a fixed direction. Observe that in the latter context, shadow prices can be 
determined as a special case. 


Perturbation of the Cost Vector 


Consider the following problem: 


Minimize cx 
subject to Ax 
x 


b 
0. 


IV Il 


Assume that B is an optimal basis. Suppose that the cost vector ¢ is perturbed 

along the cost direction c’, that is, c is replaced by e+ Ac’ where X > 0. We are 

interested in finding optimal solutions and corresponding objective values as a 

function of 4 = 0. Decomposing A into [B, N], ¢ into (cg,c,), and ec’ into 
(cz, cr), we get 

z—(€p +Acp)Xp -(Cy + Achy )Xy 

Bx, +Nxy 


| 
i ao) 


Updating the tableau and denoting cpy ; by z', we get 
z+ S| (z;-¢,;)+Az,-c) k&; = cpb+Ac'gb 
zl pg | ky 2 
Xp+ Dy pty = dD, 
jeJ 

where J is the set of current indices associated with the nonbasic variables. The 
current tableau has 4 = 0 and gives an optimal basic feasible solution for the 
original problem without perturbation. We would first like to find out how far 


we can move in the direction ec’ while still maintaining optimality of the current 
point. Let S = {j:(zj —c;) > 0}. If S=, then the current solution is optimal 


for all values of A > 0 (why?). Otherwise, we calculate A as follows: 
P -(z;-c;)| -(z,- 
A = minimum see Bea! Sak 24 (6.23) 


jeS Hat Zi — Ch 


Let A, = A. For 2€ (0, 2, ] the current solution is optimal and the optimal objective 


value is given by cgb+Ac,b=cpB 'b+AcgB 'b. For 2€[0, 4], the objective 
row coefficients in the simplex tableau are replaced by (z;—c;) + Az) Cys 
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AtA=A,, x; iS introduced into the basis (if a blocking variable exists). After the 


tableau is updated, the process is repeated by recalculating S and / and letting 
Ay =A. For 2.€[A,,4)] the new current solution is optimal and its objective 
value is given by ¢gb+Ac,b=c,B 'b + Ac’,B 'b where B is the current 
basis. The process is repeated until S becomes empty. If there is no blocking 
variable when x; enters the basis, then the problem is unbounded for all values 
of A greater than the current value. 


Example 6.17 
Consider the following problem: 
Minimize -x, — 3x 
subjectto x + 4X < 6 
xX + 2X9 < 6 
x); x4 > 0. 


It is desired to find optimal solutions and optimal objective values of the class of 
problems whose objective function is given by (—1 + 2A, -3 + A) for 2 > 0; that 
is, we perturb the cost vector along the vector (2,1). First we solve the problem 
with A= 0 where x3 and x4 are the slack variables. The optimal tableau for A = 


0 is given by the following: 


Z xX XQ x3 x4 RHS 


ea a 


i 0 393-468 2 
O° a GBs AB 4 


In order to find the range over which this tableau is optimal, first find 
cB 'N-cly: 


cgB'N-cy = ¢(y3,¥4)—(c4,c4) 
2/3 -1/3 


_ : (6.24) 
= 2.0} 13 3 (0.0) = (5/3,-1/3). 


Therefore, S = {3}, and from Equation (6.23), 4 is given by 


spits sibs = 63) _ 25/3) _| 
23-03 5/3 
Therefore, A, =1 and for A €[0,1] the basis [a,,a,] remains optimal. The opti- 
mal objective value z(/) in this interval is given by 


z(A) = egb+Ac'zb 
-14+40.0(3) =-14+82. 
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Noting Equation (6.24), the objective row coefficients of the nonbasic variables 
x3 and x4 are given by 


(23 —¢3) + A(zh — ch) =-5/3+ (5/3)A 
(24 —c4) + A(z, -—c,) = -2/3 -(1/3)A. 


Hence, the optimal solution for any / in the interval [0,1] is given by the 
following tableau: 


At A= 1 the coefficient of x3 in row 0 is equal to 0 and x3 is introduced in the 
basis leading to the following new tableau: 


We would now like to find the interval [1, A, ] over which the foregoing tableau 
is optimal. Note that 


3/2 
2, —Cy =CBy, —¢ -0-9( fa) =5/2 


—1/2 
244 = ¢p¥4 ~€4=(0-3)( 4/9 |-O=-32 


roo as ' 3/2 
2) —¢, =CBy) —¢ on 5)-2 =~—5/2 


, U , , -l 2: 
Z4—Cg =CRY4 —C4 = c.nf 2)" = 1/2. 
Therefore, the objective row coefficients for the nonbasic variables x, and x4 
are given by 
(21 — C1) + A(zj — cj) = 5/2 -(5/2)A 


(24 —C4) + A(z —c4) = -3/2 + (1/2) A. 


Therefore, for A in the interval [1, 3] these values are nonpositive and the basis 
consisting of a3; and ay is optimal. Note that the value 4 = 3 can also be 


determined as follows: 


Duality and Sensitivity Analysis 315 


=3. 


S={4} end 2 =—@4— a) _ 3/2 
Z4—C4 1/2 


Over the interval [1, 3] the objective function z(A) is given by 


2(A) = epb+Ac'zb 


3 3 
(0, (3 |+ A(0, of) =-943/. 


Hence, the optimal tableau for 4 € [1, 3] is given as follows: 


Il 


4 x x x X4 RHS 
z 5/2-(5/2)2 0 0 -3/24+(/2)4 
% 3/2 0 1 ~1/2 3 
x» ~1/2 1 0 1/2 3 


At A = 3 the coefficient of x4 in row 0 is equal to zero, and x, is introduced in 
the basis leading to the following tableau: 


We would like to calculate the interval over which the foregoing tableau is 
optimal. First, we compute 


Z| —¢] = Cpyy — cj =—2 


2) —Cy =CRY2 —Cy =-1. 
Therefore, S=@ and hence, the basis [a,,a4] is optimal for all A €[3,<]. 
Figure 6.7 shows the optimal points and the optimal objective as a function of A. 
Note that this function is piecewise linear and concave. In Exercise 6.65 the 


reader is asked to show that this is always true. The breakpoints correspond to 
the values of A at which alterative optimal solutions exist. 


Perturbation of the Right-Hand-Side 


Suppose that the right-hand-side vector b is replaced by b + Ab’ where A> 0. 
This means that the right-hand-side is perturbed along the vector b’. Since the 
right-hand—side of the primal problem is the objective of the dual problem, 
perturbing the right-hand—side can be analyzed as perturbing the objective 
function of the dual problem. We shall now handle the perturbation directly by 
considering the primal problem. Suppose that we have an optimal basis B of the 
original problem, that is, with A = 0. The corresponding tableau is given by 


z+(cgB IN-ecy)xy = cpB'b 
Xp +B Nx, = B'b, 
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where cpB 'N-cy <0. If b is replaced by b + Ab’, the vector cpB 'N-cy 
will not be affected; that is, dual feasibility will not be affected. The only change 
is that B-'b will be replaced by B !(b+ Ab’), and accordingly, the objective 
value becomes ¢ pB | (b+ Ab’). As long as B | (b+ Ab’) is nonnegative, the 


current basis remains optimal. The value of 4 at which another basis becomes 
optimal, can therefore be determined as follows. Let S = {i: b <0} where b’ = 


B 'b’. If S =@, then the current basis is optimal for all values of A > 0. Other- 
wise, let 


i= nim | ates (6.25) 
ieS i —b,. 


Let A, = A. For A€ [0, A,] the current basis is optimal, where xz = B (b+ Ab’) 
and the optimal objective value is ¢ 3B | (b+ Ab’). At A, the right-hand-side is 
replaced by B|(b+ Ab’), xg, is removed from the basis, and an appropriate 


variable (according to the dual simplex method criterion) enters the basis. After 
the tableau is updated, the process is repeated in order to find the range [A,, A, ] 


over which the new basis is optimal, where A, = A from Equation (6.25) using 


the current basis. The process is terminated when either S is empty, in which 
case the current basis is optimal for all values of 2 greater than or equal to the 
last value of A, or else when all the entries in the row whose right-hand-side 
dropped to zero are nonnegative. In this latter case no feasible solutions exist for 
all values of A greater than the current value (why?). 


Example 6.18 


Consider the following problem: 


Optimal 
basic 
solutions 


Figure 6.7. Optimal objective values and optimal solutions as a function of 2. 
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Minimize -x,; —- 3x 

subjectto x, + Xx, < 6 
—x, + 2x, < 6 
xX xX, 2 0. 


It is desired to find the optimal solution and optimal basis as the right-hand-side 


is perturbed along the direction G} that is, if b = (°) is replaced by b + Ab’ 


= (2) + (7) for 2 > 0. The optimal solution with 4 = 0 is shown below, where 
x3 and x4 are the slack variables: 


Z XM xX x3 x4, RHS 


2 en ee ee 
x 1 0 2/3 -Y3 2 
Xp 0 I 1/3 3 4 


To find the range over which the basis is optimal, we first calculate b’ 
=, ot _| 2/3 -1/3][-1]_[-1 
anes v'=|7 /3 {| 1) | Oy 
Therefore, S= {1}, and from Equation (6.25) 4, is given by 
A = eal = = = 2. 
a ies el 


Therefore, the basis [a;,a,] remains optimal over the interval [0, 2]. In particu- 
lar, for any A € [0,2] the objective value and the right-hand-side are given by 


z(A) = epb+ Acpb’ 

= c1-3(Z}+ac-9(G]--14+4 
2 -] 2-4 
(a) o}(707): 


and the simplex tableau appears as follows: 


b+Ab’ 


Z XX  % x3 x4 RHS 
: 
x 2-A 
*2 


AtA=2, Xp, =% drops to zero. A dual simplex pivot is performed so that x 


leaves the basis and x4 enters the basis leading to the following tableau: 
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To find the range [2, A, ] over which this tableau is optimal, we first find b and b’: 
~ 34 — |{-2 1]/6]_|-6 
eae | ole al 
—2 1}]/-1]_] 3 
10}/ 1] |-1/ 


by 6 
aa ua 
=Dy. AHI) 
For A in the interval [2, 6] the optimal objective value and the right-hand-side 
are given by 


b’ = Bb’ 


Therefore, S = {2} and A, is given by 


z(A) = cpb+ Acpb’ 
(0, (+400 -»( 4) 218432 


bolas) 


The optimal tableau over the interval [2, 6] is depicted below: 


b+Ab' 


X x4 RHS 
z ee 


1 6 + 34 
0 6-A 
At A= 6, x) drops to zero. Since all entries in the x2 row are nonnegative, we 


stop with the conclusion that no feasible solutions exist for all A > 6. Figure 6.8 
summarizes the optimal bases and the corresponding objective values for A > 0. 
Note that the optimal objective value as a function of A is piecewise linear and 
convex. In Exercise 6.66 we ask the reader to show that this is always true. The 
breakpoints correspond to the values of 4 for which alternative optimal dual 
solutions exist. 


Comment on Deriving Shadow Prices via a Parametric Analysis 
Observe that parametric analysis can be used to ascertain the structure of the 
optimal value function z* (b;) (see Equation (6.4)) as a function of b;, in the 


neighborhood of the current value of b;, for any i€ {l,...,m}. Accordingly, we 
can then determine the right-hand and left-hand shadow prices with respect to 


Duality and Sensitivity Analysis 319 


Optimal 


bases 


No feasible 
solutions 
exist 


Figure 6.8. Optimal objectives and bases as a function of /. 


b, as the respective right-hand and left-hand derivatives of a (b;) at the current 
value of b;, where the former value is taken as infinity in case an increase in }; 
renders the primal problem in Equation (6.1) infeasible. 

More specifically, consider determining the right-hand shadow price 
with respect to b;, for some ie {1,...,m}. In this case, the right-hand-side b is 
replaced by b + Ab’, where b’ = e;, the ith unit vector. Accordingly, we can 
now perform the foregoing parametric analysis until we arrive at a tableau that 
remains optimal as 4 increases from zero (up to some positive level) or else, we 
detect unboundedness of the dual (infeasibility of the primal) as 4 increases 
from zero. In the former case, the right-hand shadow price is given by 
w, =(c 2B); for the corresponding current tableau, and in the latter case, it is 
infinite in value. In a similar manner, we can compute the left-hand shadow 
price as the w; value corresponding to the tableau that remains optimal as 4 
increases from the value of zero, where the right-hand—side is now perturbed 
according to b — Ae;. Exercise 6.70 asks the reader to illustrate this approach. 


EXERCISES 


[6.1] Use the standard form of duality to obtain the dual of the following 
problem. Also verify the relationships in Table 6.1. 


Minimize C)xX; + CoX + C3X3 
subject to Ay 1X] sb Aj2X2 a Aj3X3 2 b; 
A2)X + Ag X. + Ap3X3 < by 
A3)X} + Ag2X2 + Ag3%3 = b3 
x, 2 0 
X92 < 0 
X3 unrestricted. 


[6.2] Give the dual of the following problem: 
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Maximize —2x, + 3x, + 5x3 


subjectto —2x, + X% + 3x, + x4 2 5 
2x, + x4 = 4 
—2Xx4 + Xz + X%y < 6 
xX < 0 
x7, x3 > 0 
x4 unrestricted. 
[6.3] Consider the following problem: 
Maximize -x, + 3x 
subject to 2x, + 3x, S$ 
x} — 3X, => 3 
Xx, xX, 2 0. 


a. Solve the problem graphically. 

b. State the dual and solve it graphically. Utilize the theorems of duality 
to obtain the values of all the primal variables from the optimal dual 
solution. 

[6.4] Solve the following linear program by a graphical method: 
Maximize 3x, + 3x, + 21% 


subject to 6x, + 9x, + 25x, < 15 
3x, + 2x, + 25x, < 20 
X, xX, % 2 0. 


(Hint: Utilize the dual problem.) 
[6.5] Consider the following problem: 
Maximize 10x, + 24x, + 20x, + 20x4 + 25x; 


subjectto x + X. + 2x, + 3xy + 5x5 < 19 
2x, + 4x, + 3x, + 2x4 + X5 S$ 57 
xX, Xp, x3, X4, X, 2 0. 


a. Write the dual problem and verify that (w,,w)) = (4, 5) is a feasible 


solution. 
b. Use the information in Part (a) to derive an optimal solution to both 
the primal and the dual problems. 


[6.6] Consider the following problem: 
Minimize 2x, + 15x. + 5x, + 6x4 


subjectto x + 6x, + 3x, + x4 2 2 
—2 x, i 5X, — 4x, + 3x4 < -3 
xX, Xp, x3, xy 2 0. 


a. Give the dual linear problem. 
Solve the dual geometrically. 

c. Utilize information about the dual linear program and the theorems 
of duality to solve the primal problem. 


[6.7] Consider the following linear programming problem: 


Maximize 2x, + 3x, + 5x3 
subjectto x, + 2x, + 3x3 
xy oF 2x5 + 2X3 
+e *2> es 


MIA IA 


8 
6 
0 
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a. Write the dual problem. 

b. Solve the foregoing problem by the simplex method. At each itera- 
tion, identify the dual variable values and show which dual con- 
straints are violated. 

c. At each iteration, identify the dual basic and nonbasic variables, 
along with the corresponding 3 x 3 dual basis. 

d. Show that at each iteration of the simplex method, the dual objective 
is “worsened.” 

e. Verify that at termination, feasible solutions of both problems are at 
hand, having equal objective values and with complementary slack- 
ness holding. 


[6.8] The following simplex tableau shows the optimal solution of a linear 
programming problem. It is known that x, and x; are the slack variables in the 


first and second constraints of the original problem. The constraints are of the < 


type. 


a. Write the original problem. 

b. What is the dual of the original problem? 

c. Obtain the optimal solution of the dual problem from the tableau. 

[6.9] The following refer to a primal—dual (min—max) pair P and D of linear 
programming problems in canonical form. Provide a brief explanation with your 
answers. 

a. Ifa basic solution to the primal is infeasible and has an objective 
value less than the optimal value, then the associated complementary 
dual basic solution is feasible. True or False? 

b. For the linear program: Minimize{x, : 2x, —x2 20, -2x, + 3x, 2 -6, 
x > 0}, consider the basic feasible solution with a basis comprised of 
the columns of x, and the slack variable in the second constraint. 
Give the associated complementary dual basic solution. What can 
you say about this pair of primal—dual basic solutions? 


c. If P has alternative optimal solutions and if w” is any optimal basic 
feasible solution for D, then w” must be degenerate. True or False? 


d. Let z* be the common (finite) optimal value of P and D. Suppose 
that x is a basic infeasible solution to P whose complementary dual 
basic solution is feasible. Is it possible that the common objective 
value of this pair of primal-dual basic solutions is z*? 

e. If P is unbounded, it is possible to change its right-hand-side and 
make it have a finite optimum. True or False? 

f. Referring to Figure 6.4a, suppose that —c is aligned along —A). 


Consider the basic solution with basis B = [a,,a4,a5]. Comment on 
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the corresponding pair of complementary primal and dual basic solu- 
tions with respect to feasibility, optimality, and degeneracy. 
[6.10] Consider the problem: Minimize z subject to z — ex = 0, Ax =b, x > 0. 
a. State the dual. 
b. At optimality, what will be the value of the first dual variable? 
Explain. 
[6.11] Prove that if a given basic feasible solution to some linear programming 
problem is optimal, the same basic vectors will yield an optimal solution for any 
right—hand-side vector that lies in the cone spanned by these basic vectors. 
[6.12] Consider the problem: Minimize ex subject to Ax = b, x > 0. Let B be a 
basis that is neither primal nor dual feasible. Indicate how you can solve this 
problem starting with the basis B. 
[6.13] Consider the problem: Minimize cx subject to Ax = b, x > 0 where m = 


n,¢= b' and A= A’. Show that if there exists an xp such that Axg =b, x9 >0, 


then Xq is an optimal solution. (Hint: Use duality.) 


[6.14] Consider the following bounded variables linear program: 


Maximize x + X 

subject to —2x, + x, S$ 2 
xX — XxX < 0 
—a2< xy < 2 
-l< x4 <2; 


a. Solve the problem graphically in the (x,, x.) space. 

b. Give all optimal basic feasible partitions. (Specify sets of basic and 
nonbasic variables at optimality.) 

c. For the extreme point (x, x.) = (0, 2), construct the bounded vari- 
ables simplex tableau and perform one iteration. Is the resulting 
tableau optimal? 

d. Graphically verify whether the following is true or false. Starting at 
the point where the slack from the second constraint and x are 


nonbasic at their lower bounds, if one introduces x, into the basis, 


then the resulting basic feasible solution is optimal. 

e. Write the dual to the foregoing problem by associating a dual 
variable with each of the six inequality constraints. 

f. Using the graph of Part (a), compute the set of dual optimal solutions 
and determine why or why not the dual has alternative optimal solu- 
tions. 

g. Graphically add the constraint x, + x. <4 to the problem. Is there a 


degenerate optimal dual basic solution? Is there a nondegenerate 
optimal dual basic solution? 


[6.15] Consider the problem: Minimize cx subject to Ax = b, 2<x <u, where @ 
and u are finite. 

a. Give the dual. 

b. Show that the dual always possesses a feasible solution. 
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c. Ifthe primal problem possesses a feasible solution, what conclusions 
would you reach? 


[6.16] Show directly that if the primal problem: Minimize ex subject to Ax > b 
and x > 0 has no feasible solutions, and if the dual problem has a feasible 
solution, then the dual problem is unbounded. (Hint: Use Farkas’ Lemma. If the 
system Ax > b and x > 0 has no solution, then the system wA < 0, w> 0, wb > 0 
has a solution.) 


[6.17] Show by duality that if the problem: Minimize cx subject to Ax = b and 
x > 0 has a finite optimal solution, then the new problem: Minimize cx subject 
to Ax = b’ and x > 0 cannot be unbounded, no matter what value the vector b’ 
might take. 


[6.18] The Sewel Manufacturing Company produces two types of reclining 
chairs for sale in the Northeast. Two basic types of skilled labor are involved— 
assembly and finishing. One unit of the top—of—the—line recliner requires 2 hours 
of assembly, | hour of finishing, and sells for a profit of $25. A unit of the 
second-line recliner requires 1 hour of assembly, 1/2 hour of finishing, and 


sells for a profit of $15. Currently there are 120 assembly hours and 85 finishing 
hours available to the company. The company is involved in labor negotiations 
concerning salary modifications for the coming year. Provide the company with 
indications of the worth of an hour of assembly worker’s time and an hour of 
finishing worker’s time. 


[6.19] Use the main duality theorem to prove Farkas’ Lemma. [Hint: Consider 
the following pair of primal and dual problems: 


Minimize 0x Minimize wb 
subject to Ax = b subjectto wA < 0 
x > 0. w unrestricted. ] 


[6.20] Show that if a set of constraints is redundant, then the corresponding dual 
variables can only be specified within a constant of addition (that is, if one dual 
variable in the set is changed by an amount @, then all dual variables in the set 
would change by appropriate multiples of @). 


[6.21] In Figure 6.4b suppose that —c is aligned along —Aj. In terms of dual 


optimal solution values, derive the right-hand and the left-hand shadow prices 
with respect to perturbations in 5,, 5), and 53. 


[6.22] For the example in Exercise 1.28, what dual variable in the optimal 
linear programming solution will yield the equilibrium price p*? Interpret this 
using the figure of Exercise 1.28 that shows the supply—demand interaction. 


[6.23] Consider a pair of primal and dual linear programs in standard form. 
a. What happens to the dual solution if the Ath primal constraint is 
multiplied by a nonzero scalar 4? 
b. What happens to the dual solution if a scalar multiple of one primal 
constraint is added to another primal constraint? 
c. What happens to the primal and dual solutions if a scalar multiple of 
one primal column is added to another primal column? 
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[6.24] Consider the problem: Maximize ex subject to Ax = b, x > 0. Let z eer 
yy» and b; be the updated tableau entries at some iteration of the simplex 


algorithm. Indicate whether each of the following statements is true or false. 
Discuss. 


a. dij = 


2 


Dual feasibility is the same as primal optimality. 

d. Performing row operations on inequality systems yields equivalent 
systems. 

e. Adding artificial variables to the primal serves to restrict variables 
that are really unrestricted in the dual. 

f. Linear programming by the simplex method is essentially a gradient 
search. 

g. A linear problem can be solved by the two-phase method if it can be 
solved by the big—M method. 

h. There is a duality gap (difference in optimal objective values) when 
both the primal and the dual programs have no feasible solutions. 

i. Converting a maximization problem to a minimization problem 
changes the sign of the dual variables. 

j. A linear program with some variables required to be greater than or 

equal to zero can always be converted into one where all variables 

are unrestricted, without adding any new constraints. 


[6.25] Let x” be an optimal solution to the problem: Minimize ex subject to alx 


= b;, i=1,...,m,x = 0. Let w’ be an optimal dual solution. Show that x" is also 
an optimal to the problem: Minimize (¢ —wya* )x subject to alx = b;, i= 1,..., m, 
i#k,x>0, where Wy is the kth component of w*. Discuss. 


[6.26] Show that discarding a redundant constraint is equivalent to setting the 
corresponding dual variable to zero. 


[6.27] Two players are involved in a competitive game. One player, called the 
row player, has two strategies available; the other player, called the column 
player, has three strategies available. If the row player selects strategy i and the 
column player selects strategy j, the payoff to the row player is cj and the 


payoff to the column player is —cj. Thus, the column player loses what the row 


player wins and vice versa; this is a two-person zero—sum game. The following 
matrix gives the payoffs to the row player: 


1 2 


3 
1 [2170] 
ca ben eR 
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Let x,, x2, and x, be probabilities with which the column player will select the 
various strategies over many plays of the game. Thus x, + x. + x3 =1, 4, %, 
x3 > 0. If the column player applies these probabilities to the selection of her 


strategy for any play of the game, consider the row player’s options. If the row 
player selects row 1, then her expected payoff is 2x, — x». If the row player 


selects row 2, her payoff is —3x, + 2x. + x,. Wishing to minimize the maximum 


expected payoff to the row player, the column player should solve the following 
linear program: 


Minimize Zz 
subjectto x, + Xx, + % = 1 
2x) — Xx < Zz 
—3x, + 2x) + % Sz 
Xs Xp, % 2 0 


Z unrestricted. 


Transposing the variable z to the left-hand-side, we get the column player’s 
problem: 


Maximize z 


subject to My + X% + x = | 
Z-— 2x, + XX 2 0 
z+ 3x, - 2x, - »&% 2 0 
X1, X>, x3 2 0 
Zz unrestricted. 


Give the dual of this linear program. 

b. Interpret the dual problem in Part (a). (Hint: Consider the row 
player’s problem.) 

c. Solve the dual problem of Part (a). (Hint: This problem may be 
solved graphically.) 

d. Use the optimal dual solution of Part (c) to compute the column 
player’s probabilities. 

e. Interpret the complementary slackness conditions for this two- 

person zero-sum game. 


[6.28] The following is an optimal simplex tableau for a maximization problem 
having all < constraints: 


» 


a. Give the optimal solution. 
b. Give the optimal dual solution. 
c. Find éz/6b,. Interpret this number. 
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g. 


h. 
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Find 6x, / xg. Interpret this number. 


If you could buy an additional unit of the first resource for a cost of 
5/2, would you do this? Why? 

Another firm wishes to purchase one unit of the third resource from 
you. How much is such a unit worth to you? Why? 

Are there any alternative optimal solutions? If not, why not? If so, 
give one. 

Find the objective value @. 


[6.29] The following are the initial and current tableaux of a linear program- 
ming problem: 


me aos p 


Zz x} x2 x3 X4 X5 X6 x7 RHS 


Z. 
X6 
x7 
yx) x4 x, RHS 
ae I 0 0 1/77 ah —50/7 | 60/7 | 
-1/7 : 1-2/7 ya -1/7 4/7 oo 
-12/7 0 -3/7 8/7 -5/7 13/7 | 4/7 
Find a, b, and c. 
Find B™, 


Find 0x /0xs. 

Find 0x3 /@by. 

Find 0z/0x¢. 

Find the complementary dual solution. 


[6. 30) § Solve the following linear program by the dual simplex method: 


Minimize 2x, + 3x, + 5x; + 6x4 


subjectto 2x, + 2x9 + 3x3 + x4 2 3 
xX + yy -— % + 3x4 < -3 
X, Xo, x3, Xy 2 0. 


[6.31] Solve the following problem by the dual simplex method: 


Maximize —4x, - 6x. — 5x; 
subject to 2x, + 3x, 2 3 
3x9 + 2x3 > 6 
X> Xp, x, 2 0. 


[6.32] Consider the following problem: 


a. 


Minimize 3x, = 5x _ x3 + 2X4 = 4x; 

subjectto x, + 2x. + 4x3 + 3xy + X5 < 6 
“Hy - XX + 2x3 + Xg -— XS oars | 
XM X2, x3, X4, Xx, 2 0 


Give the dual problem. 
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b. Solve the dual problem using the artificial constraint technique. 
c. Find the primal solution from the dual solution. 
[6.33] Consider the following linear programming problem: 


Maximize 2x, — 3x 

subjectto x) + X 2 
3x, + % < 6 
X> X 2 0. 


You are told that the optimal solution is x, = 3/2 and x) = 3/2. Verify this 


statement by duality. Describe two procedures for modifying the problem in 
such a way that the dual simplex method can be used. Use one of these 
procedures for solving the problem by the dual simplex method. 


[6.34] Apply the dual simplex method to the following problem: 


Minimize 2w, +7; 


subjectto -1/4w, —- 1/2w, < -3/4 
8w, + 12W, < 20 

w, + W2w. - w; < -1/2 

-9w, - 3w, < 6 

Ww. W, Ww, 2 0. 


[6.35] Suppose that an optimal solution to the problem: Minimize ex subject to 
Ax > b, x > 0, exists. Prove the following complementarity theorem. 
a. A variable is zero for all primal optimal solutions if and only if its com- 
plementary dual variable is positive for some dual optimal solution. 
b. A variable is unbounded in the primal feasible set if and only if its 
complementary dual variable is bounded in the dual feasible set. 
c. A variable is unbounded in the primal optimal set if and only if its 
complementary dual variable is zero for all dual feasible solutions. 
d. A variable is positive for some primal feasible solution if and only if 
its complementary dual variable is bounded in the dual optimal set. 


[6.36] Consider the following problem: 


P: Minimize 2x, -— 4x, 


subjectto x, + x 2 2 
-— xX 2 -5 
x} mo) 2 0. 


Solve P graphically. 

Give the dual of P. Solve the dual graphically. 

c. Illustrate the theorem of the previous exercise for each primal and 
dual variable, including slacks. 


asf 


[6.37] Show that the dual simplex algorithm is precisely the primal simplex 
algorithm applied to the dual problem. Be explicit. 


[6.38] Show that, in the dual simplex method, if b, <0 and Jy 0 for = |,..., 


n, then the dual is unbounded (and the primal is infeasible) by constructing a 
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suitable direction. (Hint: Consider w = ¢gB™! + AB”, where B’ is the rth row of 
B!) 

[6.39] Given the primal basis in Equation (6.9), show that the complementary 
dual basis in Equation (6.10) is invertible and corresponds to the dual solution 
(W,o.3W_) = CpB I. 


[6.40] In Section 6.5 we showed that the complementary dual basis matrix for 
the linear program in standard form is given by 


Give the complete starting dual tableau. 

. Give the inverse of this basis matrix. 

c. Use the result of Part (b) to develop the dual tableau associated with 
this basis matrix. 


a 


[6.41] Provide the details and the justification for the lexicographic method and 
Bland’s method discussed in Section 6.4 for preventing cycling in the dual sim- 
plex method. 


[6.42] Apply the perturbation technique to the dual simplex method to ensure 
finiteness. Specifically consider the following perturbed problem: 


Minimize (e+€)x 
subject to Ax 


b 
x 0. 


IV Il 


where ¢= (e! 2 y€") is a vector formed from the powers of a positive scalar 


é. Compare the derived procedure with the one produced by the lexicographic 
method developed in Exercise 6.41. 


[6.43] Suppose that the artificial constraint technique (with slack x,,)) is 


utilized to find a starting dual solution. Show that if this constraint is tight at 
optimality and z,,,; —¢,,, <0, then the primal problem is unbounded. Explain 
in detail the case when this constraint is tight at optimality, but z,.) —¢,4, = 0. 
How large should M be in order to reach these conclusions? IIlustrate your 
answer using the problem to minimize x, — x2 subject to —x, + x» < 1 andx>0. 


[6.44] Show that the artificial constraint technique applied to the primal 
problem is precisely the single artificial variable technique (of Chapter 4) with 
the big—M method applied to the dual problem and vice versa. (Hint: Consider 


the dual of: minimize 0x, + (ey —cpB 'N)xy subject to xp + B 'Nxy = Bb, 
XR, Xn = 0.) 


[6.45] Apply the primal—dual algorithm to the following problem: 
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Maximize 5x, + 2x. + x3 + 4x4 + 6Xs 

subject to 3x, + 5x. - 6x3; + 2x, + 4x5 = 25 
X + 2X, + 3x3 -— 7xq + 6x5 2 2 
9x, -— 4x. + 2x, + S5xqg - 2x5 = 16 
xX, X2, x3, X4, xy 2 0. 


[6.46] Solve the following problem by the primal—dual algorithm: 


Minimize 2x, + 2x3 - Xx 

subjectto x) + X» + 4% + X4 < 8 
2x, — X, + 3x3 - 2x, 2 5 
X15 X25 x3; X4 > 0. 


[6.47] Apply the primal—dual method to the following problem: 
Minimize 8x, + 7x, + 4x3 + 2x4, + 6x5 + 7X6 


subjectto x, + X% + 3 = 6 
+ Xy + Xs + Xe = 5 

xX + X5 = 4 

x3 + X6 ae 2D 

X15 x2, x3, X4> X55 X6 > 0. 


[6.48] Solve the following problem by the primal—dual algorithm: 
Maximize 2x, + 6x 


subjectto x) + Xx, 2 3 
x + 2x < 3 
x, X» 2 0. 


[6.49] When is the primal—dual algorithm preferred to the dual simplex algo- 
rithm and vice versa? 


[6.50] Using the restricted (total) Phase I interpretation of the primal—dual 
algorithm, give the details of a lexicographic cycling prevention rule. (Hint: See 
Section 6.5.) 


[6.51] Suppose that at the end of the restricted primal problem we have xg > 0 
and v'a j <0 for all 7. Using Farkas’ Lemma, show that the primal problem has 


no feasible solution. 


[6.52] Consider the following optimal tableau of a maximization problem 
where the constraints are of the < type: 


SLACKS 


a. Find the optimal objective value @. 
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b. Would the solution be altered if a new activity x9 with coefficients 


(6,0,-3)' in the constraints, and price of 7, were added to the 


problem? 
c. How large can 4, (the first constraint resource) be made without 
violating feasibility? 
[6.53] Consider the tableau of Exercise 6.52. Suppose that we add the constraint 
2x, —X_ +2x3 <2 to the problem. Is the solution still optimal? If not, find a new 
optimal solution. 


[6.54] Consider the following linear programming problem and its optimal final 
tableau shown below: 


Maximize 2x) + x2 - 3 

subjectto x + 2x. + 2% < 8 
—Xy Ft xX 2X3 < 4 
x, xX, x3 2 0. 


Final Tableau: 


a. Write the dual problem and give the optimal dual variable values 
from the foregoing tableau. 

b. Using sensitivity analysis, find a new optimal solution if the 
coefficient of x2 in the objective function is changed from | to 5. 

c. Suppose that the coefficient of x in the first constraint is changed 
from +2 to 1/6. Using sensitivity analysis, find a new optimal 


solution. 

d. Suppose that the following constraint is added to the problem: x, + 
2x3 = 3. Using sensitivity analysis, find the new optimal solution. 

e. If you were to choose between increasing the right-hand-side of the 
first and second constraints, which one would you choose? Why? 
What is the effect of this increase on the optimal value of the 
objective function? 

f. Suppose that a new activity xg is proposed with unit return 6 and 


consumption vector ag = (2,1)'. Find a new optimal solution. 


[6.55] Develop in detail the rules to conduct sensitivity analyses for the 
different cases in Section 6.7 using a revised simplex implementation. Solve 
Part (d) of Exercise 6.54 using this implementation. (Hint: See the end of 
Section 6.7.) 

{6.56] A product is assembled from three parts that can be manufactured on two 
machines A and B. Neither machine can process different parts at the same time. 
The number of parts processed by each machine per hour are summarized below: 
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MACHINE A MACHINE B 


Part 1 10 8 
Part 2 16 12 
Part 3 - 25 


Management seeks a daily schedule (for 8 hrs/day) of the machines so that the 
number of assemblies is maximized. Currently the company has three machines 
of type A and five machines of type B. 

a. Solve the problem. 

b. If only one machine can be acquired, which type would you recom- 
mend and why? 

c. Management is contemplating the purchase of a type A machine at a 
cost of $100,000. Suppose that the life of the machine is 10 years and 
that each year is equivalent to 2000 working hours. Would you 
recommend the purchase if the unit profit from each assembly is $1? 
Why or why not? 

[6.57] A farmer has 500 acres of land and wishes to determine the acreage 
allocated to the following three crops: wheat, corn, and soybeans. The man—days 
required, preparation cost, and profit per acre of the three crops are summarized 
below. Suppose that the maximum number of man—days available are 5000 and 
that the farmer has $80,000 for preparation. 


CROP MAN-DAYS PREPARATION COST$ _ PROFIT $ 


Wheat 6 100 60 
Corn 8 150 100 
Soybeans 10 120 80 


a. Find an optimal solution. 

b. Assuming an 8—hour work day, would it be profitable to the farmer 
to acquire additional help at $3 per hour? Why or why not? 

c. Suppose that the farmer has contracted to deliver at least the equiva- 
lent of 120 acres of wheat. Use sensitivity analysis to find a new 
optimal solution. 


[6.58] An airline company wishes to assign two types of its aircraft to three 
routes. Each aircraft can make at most two daily trips. Furthermore, 3 and 4 
aircraft of types A and B are available, respectively. The capacity of type A 
aircraft is 140 passengers and that of type B aircraft is 100 passengers. The 
expected number of daily passengers on the three routes is 300, 700, and 220, 
respectively. The operating costs per trip on the different routes are summarized 
below: 
OPERATING COST FOR A GIVEN ROUTE 


AIRCRAFT TYPE 1 2 3 
A 3000 2500 2000 
B 2400 2000 1800 


a. Find an optimal solution of the continuous linear programming prob- 
lem. Does this solution make any sense? 
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b. Using the cutting plane algorithm of Section 6.7, find an optimal 
integer solution. 


[6.59] The tourism department of a certain country would like to decide which 
projects to fund during the coming year. The projects were divided into three 
main categories: religious, historical, and construction (hotels, roads, recreation 
centers, and so on). Three proposals A, B, and C for restoring religious sites are 
submitted, with estimated costs of $5, $7, and $4 million, respectively. Four 
proposals D, E, F, and G for the restoration of historical sites are submitted with 
respective estimated costs of $15, $12, $8, and $9 million. Finally, five proposals 
H, I, J, K, and L for constructing new facilities are submitted. These cost $2, 
$15, $22, $12, and $15 million, respectively. In order to determine the relative 
priority of these projects, experts from the tourism department developed a 
scoring model with the following scores for proposals A, B, C, D, E, F, G, H, I, J, 
K, L: 5, 6, 2, 7, 11, 1, 7, 2, 10, 9, 5, and 4, respectively. The department decides 
that at least one project of each category must be funded. Projects E and F 
represent a continuation of a plan that started during the previous year, and at 
least one of them must be funded. Furthermore, at most two historical and three 
construction projects can be chosen. Which projects should the tourism 
department fund in order to maximize the total score and not to exceed $90 
million? (Hint: Suppose that project j is chosen if x; = 1 and is not chosen if x; 


= 0. First solve the continuous linear program by the bounded simplex method 
and then add appropriate cuts.) 


[6.60] Assuming c’ =c and b’ =b, find the maximum allowable tolerances a 


and fo on variations in the objective coefficients and on the right-hand-sides 


respectively, to maintain the (optimal) basis of Exercise 6.54 as optimal. Sup- 
pose it is known that the objective coefficient for x, can vary no more than +30 


percent. How does this affect a? On the other hand, suppose it is known that 


the variation /, in the first constraint is no more than 50 percent of the variation 
> in the second constraint. How does this affect 29 ? 


[6.61] Develop in detail the rules to conduct sensitivity analyses for the differ- 
ent cases in Section 6.7 for the bounded variables simplex algorithm. Include the 
case of changing bounds on variables. To illustrate, consider Example 5.6 in 


Section 5.2 and suppose that the column of x, is changed from (1,1)' to (-1,-1)’. 


Update the optimal solution using sensitivity analysis on the given final tableau. 
Repeat assuming that the upper bound on x» is changed from 6 to 1/2. (Hint: 


See the end of Section 6.7.) 


[6.62] Develop the details of a dual simplex algorithm for the bounded 
variables simplex algorithm. Provide convergence arguments. Illustrate by using 
this algorithm to update the optimal solution for the sensitivity analysis in— 
stances of Exercise 6.61. (Hint: Choose a row of a basic variable that violates its 
upper or lower bound as the pivot row, and pick an appropriate pivot column 
where the change in the nonbasic variable upon pivoting improves the bound 
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infeasibility in this pivot row, while maintaining dual feasibility. Change this 
nonbasic variable until the basic variable in the current row reaches its nearer 
bound. Proceed with this same row if infeasibility persists, until it is resolved.) 


[6.63] Consider the following algorithm known as the Criss—Cross or the Self— 
Dual Method. Suppose that a basic solution to the problem: Minimize {ex : Ax 
=b, x > 0} is given. If this is primal feasible, proceed using the primal simplex 
algorithm. Similarly, if this is dual feasible, proceed using the dual simplex 
algorithm. Otherwise, perform the following type of iteration. If the elements in 
a primal infeasible row are all nonnegative, then stop; the primal is infeasible. If 
the elements in a dual infeasible column (with z j ~¢; > 0) are all nonpositive, 


stop; the dual is infeasible. Otherwise, perform a primal or dual pivot as follows. 
(One can alternate between primal and dual pivots.) For a primal pivot, pick a 


nonbasic (dual infeasible) column having zj;—c; > 0 and find a leaving 


variable by performing the usual minimum ratio test, but using only the primal 
feasible rows. However, pivot as usual and update the entire tableau. If no such 
pivot is possible using any dual infeasible column in this manner, try a dual 
pivot. For a dual pivot, pick a primal infeasible row and determine a pivot 
column by performing the usual (dual simplex) minimum ratio test, but using 
only the dual feasible columns. Again, if a pivot column is available, then pivot 
by updating the entire tableau. If no such pivot is possible using any primal 
infeasible row in this manner, then try a primal pivot. However, if the primal 
pivot was already found to be unperformable, then stop; no (finite) optimal solu- 
tion exists. 

a. Justify the three termination statements of the algorithm. 

b. Illustrate the algorithm using the slack variables as a starting basis in 

Example 4.3 of Section 4.2. 
c. Show that the algorithm will converge finitely (barring cycling) if a 


constraint Big x; <M is added to the problem after a finite number 


of iterations (typically after 2(n + m) iterations), where M is 
arbitrarily large, and then only the primal pivots are performed until 
dual feasibility is achieved, whence the usual dual simplex algorithm 
is used. 


[6.64] Consider Exercise 6.3. Suppose that the cost vector is modified in the 
direction (+1, -1). Using parametric analysis on the cost vector, find the 
sequence of optimal solutions. 
[6.65] Prove that parametric analysis on the cost vector in a minimization prob- 
lem always produces a piecewise—linear and concave function z(A). 
[6.66] Prove that parametric analysis on the right—hand-side vector in a mini- 
mization problem always produces a piecewise—linear and convex function z(A). 
[6.67] Consider the following problem: 

Maximize 2x] + 4x, + 5x3 


subjectto x + XxX, + 2x < 8 
Xo Xp + 3x3 < 4 
X> x5 X3 > 0 
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a. Find an optimal solution. 
b. Find a new optimal solution if the cost coefficient c. changes from 4 
to —4. 
c. Determine optimal solutions as the cost coefficient c, varies over 
the entire real line (—0, 2). 
{6.68] Consider the following optimal tableau of a maximization problem where 
the constraints are of the < type: 


SLACKS 


Construct the sequence of optimal solutions for bj =b;-@ where @ varies 
between 0 and ~. 
[6.69] Consider the following problem: 


Minimize —x, + 2x. —- 2x3 

subjectto x + Xx) + 2x3 < 6 
—xX,) + 2x, + 3x3 < 9 
X> x2; x3 2 0. 


a. Solve the problem by the simplex method. 
b. Suppose that the vector ¢ = (-1, 1, —2) is replaced by (-1, 1, -2) + 
A(2, —2, 3) where 2 is a real number. Find optimal solutions for all 
values of J. 
[6.70] Consider the following linear program, where b, = 2, by = 2, and b, = 
2/3: 
Maximize x, + 2x) 


subjectto x + 2% < Bb 
2x] ot x Ss by 

x2 S< bs 

X> xX 2 0 


a. Solve this problem by the simplex method. 
b. Using parametric analysis, determine the right-hand and the left-— 
hand shadow prices with respect to each of b,, b), and b; at their 


respective current values. 


[6.71] Consider the following primal problem: 
Minimize cx 
subjectto Ax = b 
xe X, 


where X is a polyhedral set. (Often the set XY consists of constraints that are easy 
to handle.) Associated with the foregoing primal problem is the following 
Lagrangian dual problem: 
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Maximize /(w) 
subject to w 20 
where f(w) = wb + minimum,, y (¢— WA)x. 
a. Show that if x9 is feasible to the primal problem, that is, Axy > b 
and Xg € X, and Wo is feasible to the Lagrangian dual problem, that 
is, Wo = 0, then exy > (Wo). 


b. Suppose that X is nonempty and bounded and that the primal 
problem possesses a finite optimal solution. Show that 


minimum ex = maximum /(w) 
Ax2b w20. 
xeX 


[6.72] Consider the problem: Minimize x,+2x, subject to 3x, +x, 26, 
—XytX_ $2, Xy +X $8, and x, xX 20. Let X= {x: —xy tx $2, xy +x $8, 
xX}, X42 > O}. 
a. Formulate the Lagrangian dual problem, where w is the Lagrangian 
multiplier associated with the first constraint in the above problem. 
b. Show that {w) = 6w + minimum {0, 4 — 2w, 13 — 14w, 8 — 24w}. 
(Hint: Examine the second term in f(w) in Exercise 6.71 and enumer- 
ate the extreme points of X graphically.) 
c. Plot fw) as a function of w. 
d. From Part (c) locate the optimal solution to the Lagrangian dual 
problem. 
e. From Part (d) find the optimal solution to the primal problem. 


[6.73] Prove that if K is a skew symmetric matrix (that is, K = —K’), then the 


system 
Kx > 0, x20 


possesses at least one solution x such that Kx+x > 0. (Hint: Apply Farkas’ 
Lemma to the system Kx 20, x20, e;x > 0. Repeat for each 7 and combine 


solutions by summing.) 
[6.74] Apply the result of the previous problem to the system: 


Ax-—rb20, x20 
—-wA+re>0, w20 
wb —cx > 0, r20. 


a. Use this result to derive the fundamental theorem of duality. 

b. Use this result to prove that at optimality there exists at least one pair of 
primal and dual optimal points with the property that if a variable in one 
problem is zero, then the complementary variable in the other problem is 
positive. This is called the strong theorem of complementary slackness, 
or the strict complementary slackness property. 
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c. Illustrate the strong theorem of complementary slackness geometri- 
cally. (Hint: Consider a linear program where the objective function 
contours are parallel to one of the constraints and alternative optimal 
solutions result.) 

d. When must the solution to the strong complementary slackness theorem 
not occur at an extreme point? (Hint: Consider Part (c).) 


[6.75] Consider Beale’s example of cycling discussed in Chapter 4 (Example 
4.11). Let P denote the problem in which the last constraint in this example is 
deleted. 


a. Following Example 4.11, construct a sequence of tableaux for Problem 
P that exhibit cycling. 


b. Write the dual D to Problem P and plot its constraints (in two 
dimensions). For each basic feasible solution for Problem P in the cycle 
of Part a, exhibit the corresponding complementary dual basis in your 
graphical plot. 

c. Based on geometrical observations in Part b, devise appropriate rules by 
which a traversal of bases in the dual problem would correspond to 
cycling among a sequence of bases representing a given vertex in the 
primal problem. Discuss how you might use this geometric insight to 
construct alternative examples of linear programs that exhibit the 
phenomenon of cycling. 


NOTES AND REFERENCES 


1. 


Ww 


John von Neumann is credited with having first postulated the existence 
of a dual linear program. His insights came through his work in game 
theory and economics together with a strong mathematical component. 
Many individuals have continued to develop and extend the basic duality 
theorems, notably Tucker [1960a] and A. C. Williams [1970]. Also, see 
Dantzig [1982] for his pioneering contributions to duality theory. 

For more details on shadow prices and relationships with directional 
derivatives for nondifferentiable convex functions, see Akgul [1984] and 
Bazaraa et al. [2006]. For a related application to marginal cost pricing for an 
electric utility, where a closed form derivation for the dual variables is 
presented, see Sherali et al. [1984]. 

The dual simplex method was first developed by Lemke [1954]. 

The primal—dual algorithm was developed by Dantzig, Ford, and Fulkerson 
[1956]. This development was fostered out of the work of Kuhn [1955] on 
the assignment problem. Also, see Paparrizos et al. [2003] for recent 
improvements on this approach. 

The cutting plane application in integer programming presented in Section 
6.7 is mainly for illustration. For more up-to-date information on integer 
programming methods, see Schrijver [1986], Johnson et al. [1985], Crowder 
et al. [1983], and Nemhauser and Wolsey [1998]. 

The tolerance approach to sensitivity analysis presented in Section 6.7 is 
from Wendell [1984, 1985a, b]. Also, see Freund [1985] for sensitivity 
analysis involving simultaneous changes in constraint coefficients. For a 
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discussion on using sensitivity analyses to handle the effect of uncertainty 
versus incorporating uncertainty directly within models, see Higle and 
Wallace [2003]. 

7. The Criss—Cross algorithm presented in Exercise 6.63 is from Zionts 
[1969, 1974]. 

8. Exercise 6.75 uses duality to provide insights into the geometry of cycling 
as motivated by Avis et al. [2008]. 
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SEVEN: THE DECOMPOSITION 
PRINCIPLE 


This chapter discusses the Dantzig—Wolfe decomposition technique and its 
relationships with Benders partitioning and Lagrangian relaxation techniques for 
dealing with large-scale and/or specially structured linear programming prob- 
lems. It is not unusual in a corporate management model or in a logistics model 
to produce a linear program having many thousands of rows and a seemingly 
unlimited number of columns. In such problems, some method must be applied 
to convert the large problems into one or more appropriately coordinated 
smaller problems of manageable size. The Dantzig—Wolfe, Benders, and 
Lagrangian relaxation techniques, which are all equivalent for linear program- 
ming problems, and which we simply refer to as applying the decomposition 
principle, do exactly this. 


Even if a linear program is of manageable size, some of its constraints 
might possess a special structure that permits efficient handling. In such cases, 
we would like to partition the linear program into one part having a general 

havi ial . Ati 


Vv 
more efficient solution method can be devised. Again, the decomposition 
principle can be applied to such a linear program to achieve the desired effect. 

The decomposition principle is a systematic procedure for solving large— 
scale linear programs or linear programs that contain specially structured 
constraints. The constraints are partitioned into two sets: general constraints (or 
complicating constraints) and constraints having a special structure. It will 
become apparent that it is not necessary for either set to have special structure, 
however, special structure, when available, enhances the efficiency of the 
procedure. 

The strategy of the decomposition procedure is to operate on two separate 
linear programs: one over the set of general constraints and one over the set of 
special constraints. Information is passed back and forth between the two linear 
programs until an optimal solution to the original problem is achieved. The lin- 
ear program over the general constraints is called the master problem, and the 
linear program over the special constraints is called the swbproblem. The master 
problem passes down a continually revised set of cost coefficients (or prices) to 
the subproblem, and receives from the subproblem a new column (or columns) 
based on these cost coefficients. For this reason, such a procedure is also known 
as a column generation method or a price directive algorithm. 

We shall begin by assuming that the special constraint set is bounded. 
Once the decomposition principle is developed for this case and we have dis- 
cussed how to get started, we shall relax the boundedness assumption. We shall 
also extend the procedure to problems having multiple subproblems, and exhibit 
its relationship with other procedures. 
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7.1 THE DECOMPOSITION ALGORITHM 


Consider the following linear program, where X is a polyhedral set representing 
specially structured constraints, A is an m x n matrix, and b is an m—vector: 


Minimize cx 
subjectto Ax = b 

xe X, 

To simplify the presentation, assume that X is nonempty and bounded (this 
assumption will be relaxed in Section 7.4). Since X is a bounded polyhedral set, 
then any point x € X can be represented as a convex combination of the finite 
number of extreme points of X. Denoting these points by x,,x9,....x,, any x € X 


can be represented as 


t 
ie ae 
Aj = 0, Usb act: 


Substituting for x, the foregoing optimization problem can be transformed into 
the following so—called master problem in the variables A;,/y,...,A,: 


t 
Minimize > (ex; )Aj 
j=l 


t 
subject to Y (Ax ;)A; = b (7.1) 
j=l 
t 
> A; = | (7.2) 
j=l 
A; 20, J=l.ust (7.3) 


Since f, the number of extreme points of the set X, is usually very large, attempt- 
ing to explicitly enumerate all the extreme points x,,X>,...,x,, and explicitly 


solving this problem is an onerous task. Rather, we shall attempt to find an 
optimal solution to the problem (and hence to the original problem) without 
explicitly enumerating all the extreme points. 


Application of the Revised Simplex Method 


Consider solving the foregoing master problem by the revised simplex method. 
Suppose that we have a basic feasible solution 4 = (Ag, Ay). Further suppose 


that the associated (m+ 1) x (m + 1) basis inverse B”! is known (the process of 
initialization is discussed in detail in Section 7.3). Denoting the dual variables 


corresponding to Equations (7.1) and (7.2) by w and a, we get (w, a) =¢ gb) 
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where ¢g is the cost coefficient vector of the basic variables with ¢; = ex; for 
each basic variable 4 f° The basis inverse, the dual variables, the values of the 
basic variables, and the objective function value are displayed below, where 


b-B"(P): 


BASIS INVERSE RHS 


| a) | ea 
pa 


The revised simplex method proceeds by concluding that the current solution is 
optimal or else by deciding to increase a nonbasic variable. This is done by first 
calculating 


; : ‘ 2 AX; 
Ze Ce = maximum z;—¢; = maximum oma] ‘J-ex, 
I< jst I< jst (7.4) 
= maximum WAX; +a@-C¢x,. 
l<j<t 
Since z;-¢; = 0 for basic variables, then the foregoing maximum is 
nonnegative. Thus, if z, -¢, = 0, then z; —é; <0 for all nonbasic variables and 


an optimal solution is at hand. On the other hand, if z,—¢, > 0, then the 
nonbasic variable 2, may be increased. 

Determining the index & using Equation (7.4) directly is computationally 
infeasible because ¢ is usually very large and the extreme points x ; correspond- 
ing to the nonbasic variables 2; are not all explicitly known. Therefore, an 


alternative scheme must be devised. Since X is a bounded polyhedral set, the 
maximum of any linear objective function over this set will be achieved at one 
of its extreme points. Therefore, 


maximum (wA-—c)x;+@=maximum (wA-c)x+@. 
ls jst xeX 
To summarize, given a basic feasible solution (Ag, Ay) having dual variables 
(w, @), we solve the following linear subproblem, which is “easy” because of 
the special structure of X: 
Maximize (wA-—c)x+@ 
subjectto xeX. 


Note that the objective function contains a constant. This is easily handled by 
initializing the (right—hand-side) value for z to @ instead of the normal value of 
0. Let x, be an optimal solution to the foregoing subproblem with objective 


value z, —¢,. If z, —¢, = 0, then the basic feasible solution (Ag, Ay) is opti- 
mal. Otherwise, if z, —¢, > 0, then the variable 2, enters the basis. As in the 
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revised simplex method, the corresponding column es is updated by premul- 


tiplying it by B! giving y; = B! ea Note that y, <9 cannot occur since 
X was assumed to be bounded, producing a bounded master problem. The 


updated column ies k is adjoined to the revised simplex tableau. The variable 
k 


Ap, leaving the basis is determined by the usual minimum ratio test. The basis 


inverse, dual variables, and RHS are updated by pivoting at y,,. After updating, 


the process is repeated. 

We now have all the ingredients of the decomposition algorithm, a sum- 
mary of which is given below. Note that the step of solving the master program 
provides an improved feasible solution to the overall problem whenever a non- 


degenerate pivot is performed, and that the subproblem checks whether z ; —¢; 


<0 for all 2;, or else determines the most positive z, —¢,. 


Summary of the Decomposition Algorithm 


INITIALIZATION STEP 


Find an initial basic feasible solution for the system defined by Equations (7.1), 
(7.2), and (7.3) (getting an initial basic feasible solution is discussed in detail in 
Section 7.3). Let the basis be B and form the following master array where (w, 


@) _ é,B! (recall that e = CX ; )s and b a B! (*). 


BASISINVERSE —RHS 


[et 


MAIN STEP 
1. Solve the following subproblem: 
Maximize (wA-—c)x+@ 
subjectto xeX. 
Let x; be an optimal basic feasible solution with objective value 


Ze —Cy. If zz, —¢, = 0 stop; the basic feasible solution of the last 


master step is an optimal solution to the overall problem. Otherwise, 
go to Step 2. 
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2. Let y; = B! eer and adjoin the updated column we a to the 
k 


master array. Pivot at y,, where the index r is determined as follows: 


a at {2 | 
= minimum, —: yz, >0>. 
Vrk I<i<m+l | Viz 

This updates the dual variables, the basis inverse, and the right— 
hand-side. Repeat Step | using the resulting updated master array. 


Some Remarks on the Decomposition Algorithm 


1. Note that the foregoing algorithm is a direct implementation of the 
revised simplex method, except that the calculation of z, —¢, is per- 
formed by solving a subproblem. Therefore, the algorithm converges 
in a finite number of iterations, provided that a cycling prevention 
tule is used in both the master step and the subproblem in the pres- 
ence of degeneracy. 

2. At each iteration, the master step provides (at a nondegenerate pivot 
step) a new improved basic feasible solution for the system given by 
Equations (7.1 ), (7.2), and (7.3) by introducing the nonbasic variable 
A, that is generated by the subproblem. At each iteration, the sub- 


problem provides an extreme point x; , which corresponds to an updated 
Zk —Ch . . : 
column 4 , and hence, as pointed out previously, this proce- 
k 


dure is sometimes referred to as a column generation method. 

3. At each iteration, a different dual vector is passed from the master 
step to the subproblem. Rather than solving the subproblem anew at 
each iteration, the optimal basis of the last iteration could be utilized 
by modifying the cost row. 

4. At each iteration, the subproblem need not be completely optimized. 
It is only necessary that the current extreme point x, satisfies z, — 
c, = (wA-c)x; +a@>0. In this case, A, is a candidate to enter the 
basis of the master problem. 

5. If the master constraints are of the inequality type, then we must 


check the z;—¢, for nonbasic slack variables in addition to solving 


the subproblem. For a master constraint i of the < type having an 
associated slack variable s;, we get 


Z5,— C5, = ow, 09(}-0 = Ww. 
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Thus, for a minimization problem, a slack variable associated with a 
< constraint is eligible to enter the basis if w; > 0. (Note that the 


entry criterion is w; <0 for constraints of the > type.) 


It should be clear that if there are other variable columns that are 
explicitly maintained in the master problem in addition to the A- 
variable columns or the slack columns, then like the latter, these 
columns also may be priced using the simplex multiplier vector (w, 
a) in the usual manner in order to test for their eligibility to enter the 
basis. (See Exercise 7.20.) 

Assume that the original linear program is feasible and that A is m x 
n of rank m, so that the master problem has basic feasible solutions 
with bases of size (m + 1) x (m + 1). Since there are m linearly 
independent constraints Ax = b binding at every feasible solution, we 
are searching for an optimum over some p = (n — m) dimensional 


subset of X. Hence, at an optimal extreme point solution x” to the 
original linear program, (at least) some p linearly independent 


constraints defining X must be binding. Consequently, x” lies on (at 
most) some (m — p) = m dimensional face of X. From the discussion 
of Chapter 2 it follows that since this face is itself (at most) an m— 
dimensional polytope with its vertices being a subset of the vertices 


of X, we can manage to represent x” using at most (m + 1) vertices 
of X. Hence, we will need at most (m + 1) positive A-variables at 
optimality. This of course concurs with the size of the master 
problem basis. 

On this (at most) m dimensional face of X, there may be several 


ways of representing x” in terms of at most (m + 1) extreme points 
of X. Computationally, it so turns out that one can usually achieve a 
convex combination of a set of extreme points of X that brings the 
resulting solution to within 1—5 percent of optimality fairly quickly. 
However, it has been observed on many classes of problems that the 
tail-end convergence rate in obtaining a more exact representation 
for an optimal solution can be very slow. (The algorithm need not 
produce an extreme point optimal solution for the original problem 
when alternative optima exist.) Hence, in practice, the procedure is 
frequently terminated once it comes to within 1-5 percent of 
optimality. The next discussion provides a useful construct for 
implementing such a termination criterion. In addition, Exercises 
7.42 and 7.43 describe certain stabilized column generation 
implementation techniques to improve the convergence behavior of 
the Dantzig—Wolfe decomposition method in particular, and of 
column generation procedures in general (also, see the Notes and 
References section). 
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Calculation and Use of Lower Bounds 


taal = 0. 


Because of the enormous number of variables (4; , Ay,...,4,), continuing the compu- 


Recall that the decomposition algorithm stops when maximum z 


tations until this condition is satisfied may be exceedingly time-consuming for 
large problems. 

We shall develop a lower bound on the objective value for any feasible 
solution to the overall problem, and hence, a lower bound on the optimal objective 
value. Since the decomposition algorithm generates feasible points having 
nonworsening objective values via the master problem, we have a sequence of 
nonincreasing upper bounds. Hence, we may stop when the difference between 
the objective value of the current feasible point and the available lower bound is 
within an acceptable tolerance. This may not give the true optimal solution, but 
will guarantee a good feasible solution, within any desired accuracy from the 
optimum. Consider the following subproblem: 


Maximize (wA-c)x+a@ 
subjectto xeX, 


where w is the dual vector passed from the master step. Let the optimal objec- 
tive value of the foregoing subproblem be z, —¢,. Now, let x be any feasible 


solution to the overall problem, that is, Ax = b and x € X. By the definition of 
Z% —Cx, and because x € X, we have 


(wA—c)x+@ < (z, —¢,). 
Noting that Ax = b, this inequality implies that 
ex > wAx -(z; —¢,)+@ =wh +a —(z, —€,) =E gb — (zz —C,). 
Since this is true for each x ¢ X with Ax = b, then 
minimum cx > égb —(z;, —¢;). 
Ax=b 


xeX 


In other words, ¢ pb —(z, —¢€,) is a lower bound on the optimal objective value 


for the overall problem. Note that €,b is the current best upper bound. How- 


ever, the lower bounds thus generated need not be monotone and we need to 
maintain the best (greatest) lower bound. 


7.2 NUMERICAL EXAMPLE 


Consider the following problem: 
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x, 
0je— ® > 0 = > 
0 2 x 0 x3 
0 
Figure 7.1. Representation of X by two sets. 
Minimize —2x, - x, - x3 + % 
subjectto =x + 3 < 2 
Xx + xX + 2x4, < 3 
x} < 2 
x} + 2x4 < 5 
-— %, + X% < 2 
2x3 + X4 < 6 
X15 x2, x3, X4 > 0 


Note that the third and fourth constraints involve only x, and x2, whereas the 
fifth and sixth constraints involve only x; and x4 (we shall have more to say 


about this special structure later). If we let X consist of the last four constraints 
in addition to the nonnegativity restrictions, then minimizing a linear function 
over X becomes a simple process, since the subproblem can be decomposed into 
two subproblems. Therefore, we shall handle the first two constraints as Ax <b, 


where A = I ; I b= E and the remaining constraints as x € X. Note 


that any point (x),x2,x3,%4) in X must have its first two components and its last 
two components in the respective sets X, and X> that are depicted in Figure 
71. 


Initialization Step 


The problem is reformulated as follows, where x),X>,...,X,, are the extreme 
points of X, ej =X; for j = 1,...,t, and s > 0 is the slack vector: 
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t 
Minimize > eA; 
j=l 


t 
subjectto Y (Ax,;)A;+s = b 
j=l 
t 
j=l 
A; = 0, J=1, st 
s 2 0. 


Let the starting basis consist of s and 2, where x, = (0, 0, 0, 0) is an extreme 
point of X with cx, = 0. Therefore, 


10 0 
B=|0 1 Ol. 
00 1 

b 


The vector (w,@) = ¢pB! = 0B"! =0, and b= Bo! i = eh This gives the 
following tableau, where the first three columns give (w,,w2,@) in row 0 and 


B™! in the remaining rows: 


BASIS INVERSE RHS 


Iteration 1 
SUBPROBLEM 
Solve the following subproblem: 

Maximize (wA-c)x+@ 

subjectto xeX. 
Here, (w;,w 2) = (0, 0) from the foregoing array. Therefore, the subproblem is 
as follows: 

Maximize 2x, +X) +x3-—x4+0 


subjectto xeX. 


This problem is separable in the vectors (x,,x2) and (x3,x4) and can be solved 


geometrically. Using Figure 7.1, it is easily verified that the optimal solution is 
X =(2,3/2,3,0) with objective value z, —€) =17/2. Since z, —¢) =17/2 >0, 
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then A, corresponding to xX, is introduced into the basis. The lower bound 
equals ¢pb — (z) —¢)) = 0-17/2. Recall that the best objective value so far is 0. 


MASTER STEP 


Zy —Cy = 17/2, and 


2 
_ 0113/2|_[ 5 
Ax, =|} 10 | 3 Ba 
0 
Accordingly, 
AX) | _ 
f7|- 7? 


is updated by premultiplying by B"!. This yields 


5 5 5 
y2 =B!|7/2|=1/7/2|=|7/2). 
1 1 1 


We therefore insert the column 
17/2 
k = 5 
Y2 7/2 
1 


into the foregoing array and pivot. This leads to the following two tableaux (the 
Ay column is not displayed after pivoting): 


BASIS INVERSE RHS 


The best-known feasible solution to the overall problem is given by 
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X = Ax, +x 
= (3/5)(0,0,0,0) + (2/5)(2,3/2,3,0) = (4/5, 3/5,6/5,0). 


The current objective value is —17/5. Also,(w,,w2,@) = (-17/10,0,0). 


Iteration 2 


Since w, <0, s, is not eligible to enter the basis. 


SUBPROBLEM 

Solve the following problem: 
Maximize (wA-c)x+a@ 
subjectto xeX, 


where 


wA—e=(-17/10,0)| ; , 9 |-2-4-1) = G/10,1,-7/10,-0. 


Therefore, the subproblem is as follows: 

Maximize (3/10). +x —(7/10)x3 —x4 +0 

subjectto xeEX. 
This problem decomposes into two problems involving (x,,x2) and (x3,x4). 
Using Figure 7.1, the optimal solution is x3 =(0,5/2,0,0) with objective value 
23 —€3 =5/2. Since z3—¢3 > 0, then A; is introduced into the basis. 

The lower bound is égb —(z3 —¢3) = -17/5—5/2 =—5.9. (Recall that the 

best-known objective value so far is —3.4.) 


MASTER STEP 


z3—¢€3 = 5/2, and 
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: 2Z3-¢3|. : : 
We therefore insert the column . | into the foregoing array and pivot. 
3 


This leads to the following two tableaux (the 2; column is not displayed after 
pivoting): 


BASIS INVERSE  RHS 


Zz 5/2 
A 0 0 2/5 
5) 1 0 8/5 
ai O.4 3/5 (1) 
4 
Ay 
52 
As 


The best-known feasible solution to the overall problem is given by 


AnX2 + A3X3 
(2/5) (2,3/2,3,0) + (3/5)(0,5/2,0,0) = (4/5,21/10,6/5,0). 


x 


The current objective value is —4.9. Also, (Wj, w2,@) = (—6/5,0,—5/2). 


Iteration 3 


Since w, <0, s, is not eligible to enter the basis. 


SUBPROBLEM 
Solve the following subproblem: 


Maximize (wA-c)x+a@ 
subjectto xe_X, 


where 
WA -c=(-6/5,0)/ oe 3 |--2.-b LD = 4/5,1,-5,-, 


Therefore, the subproblem is as follows: 

Maximize (4/5)x, +x —(1/5).x3 —x4 —5/2 

subjectto xeX. 
Using Figure 7. 1, the optimal solution is x4 = (2,3/2,0,0) with objective value 
24 —€4 = 3/5, and so, Aq is introduced into the basis. 
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The lower bound is given by égb —(z4 —€4) =—49/10 — 3/5 =— 5.5. Recall 


that the best-known objective value so far is —4.9. If we were interested only in 
an approximate solution, we could have stopped here with the feasible solution 
x = (4/5,21/10,6/5,0), whose objective value is —4.9. 


MASTER STEP 


Z4 —€4 =3/5, and 


ce, -[ ‘ : a = oh 


The updated column yy is given by 


50 Off 2] [2/5 
y,=B'! |Ar*|- -/5 1 —5/2|17/2|= if 
=y5-0 Ua ll 3/5 


We therefore insert the column gs *) in the foregoing array and pivot. This 
4 


leads to the following two tableaux (the A, column is not displayed after 


pivoting): 
Ag 
aE 
3/5 
BASIS INVERSE RHS 


z 
A | ¥3  -2/3 53 | 1/3 
Ag 5/3 -25/6 | 1/6 
Ay - Tae 4/2 


BASIS INVERSE RHS 


-6/5 0 —5/2 | -49/10 


Al Ws 0 0 2/5 
sy| -I/5 1 -5/2| 1/10 
Ay | 2S 0 4 3/5 


The best-known feasible solution to the overall problem is given by 
x = AnXy + A3X3 +A4X4 
= (1/3)(2,3/2,3,0) +(1/2)(0,5/2,0,0) + (1/6) (2, 3/2, 0, 0) = (1, 2,1,0). 


The objective value is —5. Also, (w,,w2,@) =(—1,—1,0). 


Iteration 4 


Since w, <0 and w, <0, neither s, nor s is eligible to enter the basis. 
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SUBPROBLEM 
Solve the following subproblem: 


Maximize (wA-—c)x+a@ 
subjectto xeX, 


where 


wA-c=(-1-D)y ae $|-C2.-L-LD = ©.0,0,-3) 


Therefore, the subproblem is as follows: 


Maximize Ox, +0x2 +0x3;—3x4+0 


subjectto xe X. 


Using Figure 7.1, an optimal solution is x5 =(0,0,0,0) with objective value 
Zs —€s = 0, which is the termination criterion. Also, note that the lower bound is 
€pb—(zs —é5) =-5-0=~-5, which is equal to the best (and therefore optimal) 
solution value known so far. 

To summarize, the optimal solution is given by (x1,x2,%3,%4) = (1,2,1,0) 
with objective value —5. The progress of the lower bounds and the objective 
values for the primal feasible solutions generated by the decomposition 
algorithm is shown in Figure 7.2. Optimality is reached at Iteration 4. If we were 
interested in an approximate solution, we could have stopped at Iteration 3, for 
example, since we have a feasible solution with an objective value equal to 
-4,9, and meanwhile are assured (by the lower bound) that there exists no 
feasible solution having an objective value less than —5.5. 

The optimal solution (x,,x2,%3,%4) =(1,2,1,0) is shown in Figure 7.3 in 
the two sets X, and X>. Note that (1, 2) is not an extreme point of X, and (1, 
0) is not an extreme point of X . Note, however, that we can map the master 
constraints 

x + X3 
xy + X2 + 2x4 


IA IA 
Wh 


into the (x,,x2) space by substituting x; = 1 and x4 = 0. This leads to the two 
restrictions x, < 1 and x, +x <3, which are shown in Figure 7.3. We see that 
(1, 2) is an extreme point of X, intersected with these two additional constraints. 
Similarly, in the (x3,x4) space, by substituting the values x, = 1 and x2 = 2, the 
master constraints reduce to x; <1 and 2x, <0. Again, (1, 0) is an extreme 


point of X intersected with these additional constraints. It is worthwhile 
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1 2 3 4 Iteration 


Progress of primal objective values 
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Figure 7.2. Progress of the primal objective values and the lower bounds. 


| i 2 bea 
y 10/3 
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Figure 7.3. Illustration of the optimal solution. 


noting that the decomposition algorithm may not provide an optimal extreme 
point of the overall problem if alternative optima exist. The reader may refer to 
Exercise 7.13. 


7.3 GETTING STARTED 


In this section, we describe a method for obtaining a starting basic feasible solution 
for the master problem using artificial variables, if necessary. These artificial 
variables are eliminated by the use of Phase I or by the big—M method. If there 
is a positive artificial variable at termination, then the overall problem has no 
feasible solution. 


Inequality Constraints 


Consider the following problem: 
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t 
Minimize > (cx; )A; 


j=l 
t 
subjectto ¥ (Ax;)A; < b 
j=l 
t 
jal 
Aj 2 0, J=\, yt 


If there is a convenient x; ¢ XY with Ax, <b, then the following basis is at 
hand, where the identity corresponds to the slack vector s > 0: 


I | Ax, _, fl) -Ax 
cal Te ees Tf 


The initial array is given by the following tableau: 


BASIS INVERSE —_RHS 


Now, suppose that there is no obvious xe XY with Ax <b. In this case, after 
converting the master problem to equality form by adding appropriate slack 
variables, the constraints are manipulated so that the RHS values are nonnega- 
tive. Then artificial variables are added, as needed, to create an identity matrix. 
This identity matrix constitutes the starting basis. The two-phase or big-M 
methods can be used to drive the artificial variables out of the basis. 


Equality Constraints 


In this case, m + | artificial variables can be introduced to form the initial basis, 
and the two-phase or the big—M method can then be applied. 


7.4 THE CASE OF AN UNBOUNDED REGION X 


For an unbounded set X, the decomposition algorithm must be slightly modified. 
In this case, points in XY can no longer be represented only as a convex combina- 
tion of the extreme points, but rather as a convex combination of the extreme 
points plus a nonnegative linear combination of the extreme directions. In other 
words, xe X if and only if 
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tf 

j=l Je 

ft 

j=l 

A; 20, J=i1 ot 
H; 29, J=1, hy 


where X|,X),...,X;, are the extreme points of X, and d,,d),...,d, are the extreme 


directions of X. The primal problem can be transformed into a so—called master 
problem in the variables A,,Ay,....A, and £4, y,.... dy as follows: 


t £ 
Minimize Y (ex;)A; + & (ed;)y; 
j=l j=l 


t £ 
subject to > (Ax ;)A; + 2D (Ad; )u; =b (7.5) 
j=l j=l 
t 
LA;=!1 (7.6) 
j=l 
A,20, jH=l,...¢ 


Because ¢ and / are usually very large, we shall attempt to solve the foregoing 
problem by the revised simplex method. Suppose that we have a basic feasible 
solution for the foregoing system with basis B, and let w and a@ be the dual vari- 


ables corresponding to Constraints (7.5) and (7.6), respectively. Hence, B, 


(w,a)=¢ pB'(é p is the vector of cost coefficients for the basic variables), and 


b=B! (*) are known, and are displayed below: 


BASIS INVERSE RHS 


Pw [a | 
ee] 


Recall that the current solution is optimal to the overall problem if z;-—¢; <0 


for each variable. In particular, the following conditions must hold at optimality: 


Aj nonbasic > 022; =C} (wa) A) )-ex, = WAX, +@—ex; (7.7) 


Hj nonbasic => 022; =¢} = (w,0)( 40) -ca, = wad ; —cd ;. (7.8) 
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Since the number of nonbasic variables is very large, checking Conditions (7.7) 
and (7.8) by generating the corresponding extreme points and directions is 
computationally intractable. However, we may determine whether or not these 
conditions hold by solving the following subproblem. More importantly, as this 
subproblem is solved, we shall see that if the Conditions (7.7) or (7.8) do not 
hold, then a nonbasic variable having a positive z, —¢,, and hence eligible to 


enter the basis, will be found. 


Maximize (wA-—c)x+a@ 
subjectto xeX. 


First, suppose that the optimal objective value for this subproblem is unbounded. 
Recall that when this occurs, an extreme direction d, is found such that 


(wA —c)d,; >0. This means that Condition (7.8) is violated. Moreover, zz —¢;, 


= (wA-c)d,; > 0 and wu; is eligible to enter the basis. In this case, ee is 


updated by premultiplying by B |, and the resulting column é _ *) is inserted in 
k 


the foregoing array and the revised simplex method is continued. 
Next, consider the case where the optimal solution value is bounded. A 
necessary and sufficient condition for boundedness is that (wA-—c)d,; <0 for 


all extreme directions, and so, Condition (7.8) holds true. Now, we check 
whether Condition (7.7) holds true. Let x, be an optimal extreme point and 
consider the optimal objective value, z, —¢,, to the subproblem. If z, —¢, <0, 
then by the optimality of x,, for each extreme point x j of X, we have 


(WA —C)x; +@<(WA~C)x, +@ =z, —C SO, 


and hence, Condition (7.7) holds true and we stop with an optimal solution to 
the overall problem. If, on the other hand, z,—c¢c, >0, then A, is introduced 


into the basis. This is done by inserting the column ee k into the foregoing 
k 


array and pivoting, where y; = B! ie } Note that, as in the bounded case, if 


the master problem includes slack or other explicitly present variables, then the 


(z; —¢; )-values for these variables must be (explicitly) checked before 


deducing optimality. Also, the bounded subproblems yield lower bounds as for 
the previous case. 

To summarize, solving the foregoing subproblem leads us either to 
terminate the algorithm with an optimal solution, or else to identify an entering 
nonbasic variable. We now have all the ingredients for a decomposition 
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x2 


xy 


Figure 7.4. Hlustration of an unbounded _X. 


algorithm for the case of an unbounded set X. Example 7.1 illustrates such an 
algorithm. We ask the reader in Exercise 7.12 to write a step-by-step procedure 
for this case. 


Example 7.1 
Minimize -x,) - 2x. -— %3 
subjectto x + X2 + x3 < 12 
—Xy + XQ < 2 
—xX, + 2x2 < 8 
x3 < 3 
XI; x2; x3 > 0. 


The first constraint is handled as Ax < b, and the rest of the constraints are taken 
to define X. Note that X¥ decomposes into the two sets displayed in Figure 7.4. 
The problem is transformed into 4,,...,4, and 44,...,4 as follows: 


t £ 
Minimize > (cx; )A; + > (ed ; \ej 
j=l j=l 


t £ 
subjectto } (Ax;)A; + ¥ (Ad;)u; <b 
j=l j=l 

t 

Aa 

j=l 
A; 2 0, j=,...¢ 
Hj 20, j=l,...,2 
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Note that x, = (0, 0, 0) belongs to X¥ and Ax; =0+0+0< 12. Therefore, the 
initial basis consists of A, (corresponding to x, ) plus the slack variable s. This 
leads to the following array, where w=a@ =0: 


BASIS INVERSE —RHS 


Iteration 1 
SUBPROBLEM 


Solve the following subproblem: 


Maximize (wA-—c)x+a@ 
subjectto xe X. 


Since w=a@ =O and A =(1, 1, 1), this problem reduces to the following: 


Maximize x, + 2x7 + x3 + 0 
subject to -—x; + xX < 2 
—x, + 2x, < 8 

X3 < 3 

X15 x0, x3 2 0. 


Note that the above problem decomposes into two problems in (x,,x2) and x3. 
The optimal value of x3 is 3. The other part of the problem can be solved 
geometrically or by the simplex method, where x, and xs are the slack 
variables. The simplex method yields the following tableau: 


This problem is unbounded by noting the x, column. Suppose, however, that we 
continue by introducing x, via Dantzig’s Rule for the simplex method to obtain 
the following sequence of tableaux: 
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Now, as x4 increases by one unit, z increases by four units, x, increases by two 


units, and x» increases by one unit; that is, in the (x,,x.) space we have found 


a direction d, = (7) leading to an unbounded objective value. In the (x,,x2,*3) 


space, d, is given by (2,1,0)' (why?). Also, (wA—c)d, =4 (the negative of 


—4 in row 0 under x4) and so, 44 is introduced into the basis. 


MASTER STEP 


z—¢, =4, and 


2 
Ad, -cuso{ ils 
0 


_pi{ Adi 
nt) 


where B! is obtained from the initial array of the master problem. In particu- 


| nf 1-0 


We therefore introduce the column ea in the master array and pivot. (The 
1 
/4-column is not displayed after pivoting.) 


BASIS INVERSE _RHS Ut 


z{_o 0 | 0] 
s 1 0 12 (3) 
A 0 1 1 
BASIS INVERSE __RHS 


a ee a 


a eer 
A 0 1 1 
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Iteration 2 

w=-—4/3 and a@ =0. Because w <0, s is not a candidate to enter the basis. 
SUBPROBLEM 

Solve the following subproblem: 


Maximize (wA-c)x+a@ 
subjectto xeX. 


This reduces to the following: 


Maximize -(1/3)x, + (2/3)x. + 0 | Maximize -(1/3)x, 
subject to -x, + X% S$ 2 subject to 0<x3 <3. 
—xy + 2x. < 8 
Xy> XH 2 0. 


Here, the value @ = 0 is added to only one of the subproblems. Obviously, x3 = 
0 at optimality above. The above problem in the variables (x,,x2) is solved by 


utilizing the corresponding tableau of the last iteration, by deleting row 0 and 
introducing the new costs as follows: 


Zz x x2 x4 x5 RHS 


To put this tableau in canonical form, we multiply row 1 by 2/3 and row 2 by 
-1/3 and add to row 0 to obtain the following tableau: 


Zz Xx] 5) X4 x5 RHS 


The foregoing tableau is optimal (not unique). The optimal objective value of 
the subproblem is (z2 —¢) = 8/3 > 0, and so, A, corresponding to xz = (x, x2, 
x3) = (4, 6, 0) is introduced into the basis. A lower bound on the problem is now 
available as ¢gb — (zz —¢,) = -16 -8/3 = 56/3. 


MASTER STEP 
Z2 -€ = 8/3, and 


AX> =10 
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_ p-1{ Axo |_ [1/3 0]/10]_ 10/3 
y=wt Ar) [M oP |-|'97 


We therefore introduce 7) into the master array and pivot. (The 2,-— 
2 


column is not displayed after pivoting.) 


BASIS INVERSE RHS 


-433  -8/3 [-56/3 
3 -10/3 273 
0 i i 


Observe that the known lower bound of —56/3 matches with the revised upper 
bound above, and hence an optimum is at hand. Let us verify this nonetheless. 


Iteration 3 


Note that w= -4/3 did not alter from the last iteration. Hence, s is still not a 


candidate to enter the basis. Also, the optimal solution of the last subproblem 
remains the same (see Iteration 2). The objective value of 8/3 was obtained for 
the previous dual solution with a= 0. For a@ = —8/3 we have z3 —¢3 = 8/3 — 8/3 


= 0, which is the termination criterion, and the optimal solution is therefore at 
hand. More specifically, the optimal solution x” is given by 


* 


xX = AX. +40] 
4 2] [16/3 
= 116|+(2/3)| 1 |=| 20/31. 
0 0 0 


The optimal objective value is —56/3. 


7.5 BLOCK DIAGONAL OR ANGULAR STRUCTURE 


In this section, we discuss the important special case when the set _X has a block 
diagonal structure. In this case, X can itself be decomposed into several sets 
X1,X9,....X7, each involving a subset of the variables that do not appear in any 
other set. If we decompose the vector x accordingly into the vectors 
X1,X2,....X7, the vector ¢ into ¢),¢€>,...,¢7, and the matrix A of the master con- 


straints Ax = b into the matrices A,,A),....A7, we get the following problem: 
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Minimize ¢)X; + ¢€)X2 +...+ ¢7X7 
subject to A)x; + A5xX> +...4 Arpxr = b 
Bx, < b; 
Bx, < b, 
Brx7 < br 
X1, X75 2a xr 2 0, 


where X; = {x; :B,;x; <b;,x; 20} for i=1.,...,7. 

Problems having the foregoing structure arise frequently in network 
flows with several commodities (see Chapter 12) and in the allocation of scarce 
resources among competing activities. Problems of this structure can be solved 
by the decomposition algorithm of this chapter (see Section 7.2 and Example 
7.1). However, this block diagonal or angular structure of X can be exploited 
further, as will be discussed in this section. 

For subproblem i, x; <¢ X; if and only if 


I 


ui fj 
xX; = Ay Xij + hyd 
j=l j=l 
fi 
% ay =1 
j=l 
Ay >0, J=1L of} 
Ly 20, J= ’ li, 


where Xj, J = 1,....¢;, and dj, j=1,...,€;, are respectively the extreme points 


and the extreme directions (if any exist) of X,;. Replacing each x; by the 


foregoing representation, the original problem can be reformulated as the 
following master problem: 


T ¢, T é; 
Minimize XY 2 (exp Ag+ 2X 2 (cdy My 
i=l j=l i=l j=l 


T fj TY 

subjectto YY (Ajxs)Ay + 2D L (Ady) aj =b (7.9) 
i=l j=l i=l j=l 
i 
Pa =1, i=1,...,7 (7.10) 
rij > 0, J=,..4%,  2=1,...,T 
Hy > 0, jJ=\,. »£;, P= 1.2375 


Note the difference in the foregoing formulation and that of Section 7.2. Here, 
we allow different convex combinations and linear combinations for each sub- 
problem i and we accordingly have T convexity constraints [the constraints of 
Equation (7.10)]. This adds more flexibility, but at the same time, increases the 
number of constraints from m+ 1 tom + T. 
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Suppose that we have a basic feasible solution for the foregoing system 
with an (m + T) x (m + T) basis B. Note that each basis must contain at least one 
variable 4, from each block i (see Exercise 7.32). Furthermore, suppose that 


Bo, b =B! (7), (W,@) = (WW Qo Ap) = épB! are known, where ¢, 


is the cost coefficient vector for the basic variables (¢, =¢;x, for Ay, and 


a =¢,dj;; for 44; ). These are displayed below in a revised simplex tableau: 


BASISINVERSE __RHS 
(w,a@) 


This solution is optimal if z,; Gi <0 for each variable (naturally z, =éy > 0 


for each basic variable). In particular, the following conditions must hold true at 
optimality: 


Ay nonbasic > 0 2 2 — ¢ = WA; Xj + @; —¢;x (7.11) 


] yoty y 


Hi nonbasic > 0 > Zi aC = wA,d,,—¢,d (7.12) 


(Gy — C/G jy. 


We can easily verify whether Conditions (7.11) and (7.12) hold true or not by 
solving the following subproblem for each i = 1,..., T: 


Maximize (wA,; —¢;)x; +a; 

subject to x, € X;. 
If the optimal objective value is unbounded, then an extreme direction dj, is 
found such that (wA; —¢;)d;, > 0; that is, Condition (7.12) is violated, and pj, 
is enterable into the basis because z —¢ = (WA; —c¢;)djz > 0. If the optimal 


objective value is bounded, then automatically Condition (7.12) holds true for 
subproblem i. Let x, be an optimal extreme point solution for the subproblem. 


If the optimal objective value z,, —¢ = WA,X,, +a; —¢;x;, <0, then Condition 
(7.11) holds true for subproblem 7. Otherwise, 4, can be introduced into the 
basis. When all subproblems have z;, —¢, <0, then an optimal solution to the 
original problem is obtained. If the master problem contains other explicit 
variables including slack variables, then we must also (explicitly) check the 
(z; —¢ '; values for these variables (as we did in Section 7.1) before 
terminating. 

To summarize, each subproblem / is solved in turn. If subproblem i yields 
an unbounded optimal objective value, then an extreme direction dj, is found 


whose corresponding variable 4, is a candidate to enter the master basis. If 
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subproblem i yields a bounded optimal value and wA,x,, +a; —¢;x;, > 0, then 
an extreme point x;, is found whose corresponding variable 4, is a candidate 


to enter the master basis. If neither of these events occurs, then there is currently 
no candidate column to enter the master basis from subproblem i. If no sub- 
problem yields a candidate to enter the master basis, then we have an optimal 
solution. Otherwise, we must select one (or more) A— or p—columns from among 
the various candidates to enter the master basis. We may use the rule of 
selecting the one having the most positive zj,—C,, or just the first positive 
Zik — Cig, and so on. If we use the rule of the first positive z,, —¢,, then we may 


stop solving the subproblems after the first candidate becomes available. On 
selecting the candidate, we update the entering column, pivot on the master array, 
and repeat the process. Note that we could sequentially enter the generated 
candidate columns as necessary, each time selecting the one having the most 
positive updated value of zj,—¢, (if any such enterable column exists), 


pivoting to update the master array, and then re—pricing the remaining generated 
candidate columns. 


Calculation of Lower Bounds for the Case of Bounded Subproblems 
Let x),X>,...,X7 represent a feasible solution to the overall problem so that x; € X; 
for each i and >); A;x; = b. By the definition of zj, —¢, we have 

(wA; —€;)X; +a; < (Zix = Ci) 
or 

C;X; 2 wA,x; +a; — (Zz —Cix ). 
Summing on i, we get 

i i i i 
But >¢;x; =ex and >} A;x; =b. Thus, we get 
i i 


ex > wh+a@l1->(z, -—Cx) 
i 


or 


cx > égh —L (zig — Cx). 
i 
The right-hand-side in this inequality provides a lower bound on the problem. 
This is a natural extension of the case for one bounded subproblem presented 
earlier. 
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Example 7.2 
Minimize -2x, - x. - 3x3 - Xx4 
subjectto x + xX» + x3 + x4 S 6 
x2 + 2x3 + X4 < 4 
XM + xX < 6 
oo) <2 
- XxX, + x4, S< 3 
x3 + X4 < 5 
XH, =X, X3, x4 2 0 


The first two constraints are treated as Ax < b, and the rest of the constraints are 
taken to represent X. Note that XY decomposes into two sets, as shown in Figure 
7.5. 

The problem is transformed into the following: 


Mette i ui bh 
Minimize (x, ;)Ayy + L (€2X2;)Ad; 
j=l j=l 


° ui v7) 
subjectto > (AyX JA; +> (A2X2;)Ay; <b 
jel j=l 


2 
X11 = (4.%2) = (0,0) and x2, =(x3,x4) =(0,0) belong to X, and X>, respec- 
tively, and satisfy the master constraints. Therefore, we have a basic feasible 
solution for the overall system where the basis consists of s,, 52, 4,1, and Ap, (51 


where ¢, =(-2,-l), ¢) =(-3,-1), Ay -|§ i and A, =|} if Note that 


and s, are the slacks). This leads to the following master array: 


BASIS INVERSE RHS 
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Figure 7.5. The region X for Example 7.2. 
The first four entries of row 0 give w,, w>, @, and @, respectively. The matrix 


B! appears under these entries. 


Iteration 1 


Solve the following two subproblems: 


SUBPROBLEM 1 SUBPROBLEM 2 
Maximize (wA,—¢))x,+@ Maximize (WA, —C))X7+@% 
subjectto x, € Xj. subjectto =X» E Xp. 


Since w = (0, 0) and a = (0, 0), these reduce to maximizing 2x, +x +0 and 
maximizing 3x3;+x4+0 over the two respective regions of Figure 7.5. The 
optimal solutions are respectively given by x, =(x),x>) = (6, 0), with objective 
value 12, and X77 =(x3,x4) = (5, 0), with objective value 15. Thus, (wA, — 
€))X}2 + a = 12 and (WA, —¢2)x2) + @ =15. Therefore, 4, and A,» are both 
candidates to enter. Select 2, because z7) — C77 =15 is the most positive. Note 
that a lower bound on the problem is 0 — 12 — 15 = —27. 


MASTER PROBLEM 


222 = C99 = 15. 
Form the column 


ArX99 


1 


and note that 
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5 
jen at Ae ehs eR? | 140 

ar22" 12 1 \\0|~ 110)’ rea) a 
1 


This column is updated by premultiplying by B ! =I. We therefore insert this 
column along with zy —€) in the master array and update the current basic 
feasible solution by pivoting. (The column for 4. is not displayed after 
pivoting.) 


BASIS INVERSE RHS 


Note that w, =0, w) =-3/2, a, = ay =0. 


Iteration 2 


Because s> just left the basis, it will not be a candidate to immediately reenter. 
We next solve the following two subproblems: 


SUBPROBLEM 1 SUBPROBLEM 2 
Maximize (WA, —C,)X) +Qy Maximize (wA, —€))X> + Ay 
subjectto x, € Xj. subjectto =X) € Xp. 


These problems reduce to the following: 


SUBPROBLEM 1 SUBPROBLEM 2 
Maximize 2x; —(1/2)x, +0 Maximize 0x3 —(1/2)x4 +0 
subject to (xX) € .X}. subject to (%3,%4) € Xp. 


The optimal solutions are respectively x3 = (x,,x2) = (6,0), with objective value 
213 —€13 =(WAy —¢]) x13 +@ =12, and X53 = (%3,x4) = (5,0), with objective value 
293 —€3 =(WA —€)X23 +@ =0. Thus, there is no candidate from Subproblem 
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2 at this time and 4,3 is a candidate to enter the master basis. The lower bound 
available at this stage is -6 — 12 -0 =—18. 


MASTER PROBLEM 
Z| aa Cc 3.= 1 2: 
Form the column 
A}X}3 
1 
0 


and note that 


6 
ax. al! MNfSl_[¢ ee Nh 
M319 1ffol|” |ol’ ; fal be 

0 


Updating this column, we get 


A)X}3 
Y13 =B” 1 = 
0 


6 

0 

if 

0 

We therefore insert this column along with z,3—¢,; =12 into the master array 


and update the current basic feasible solution by pivoting as follows (the column 
for 4,3 is not displayed after pivoting): 


BASIS INVERSE _RHS A 


Note that w, = -2, w) = -1/2, a, =a, =0. 
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Iteration 3 


Since w, < 0 and w, <0, neither s; nor sj is a candidate to enter the basis. We 
next solve the following two subproblems: 


SUBPROBLEM 1 SUBPROBLEM 2 
Maximize (WA, —¢;)x;+Q; Maximize (WA, —¢)x. +a 
subject to x; € Xj. subject to =X, € Xp. 


These problems reduce to the following: 


SUBPROBLEM 1 SUBPROBLEM 2 
Maximize 0x, —(3/2)x) +0 Maximize 0x3 —(3/2)x4 +0 
subjectto — (x),.X%)) EX}. subject to (x3,.x4) € X>. 


From Figure 7.5, x)4 = (x,x2) = (0, 0), with objective value 0, and x4 = (x3,x4) 
= (0, 0), with objective value 0 are optimal solutions. Thus, (WA, — ¢ )X,4 + @ = 
(WA, —Cy)X24 + @ =0, and an optimal solution has been found. The lower and 


upper bounds now match. From the master problem, an optimal solution x” is 
Aiki +A3%13 


Cs) 
2(0}*3(0)-0) 
) = Ay 1X1 + AnX2 
= 3(0}+3(o)-(o}: 


Therefore, x” = (X1,%2,%3,%4) = (4, 0, 2, 0) with objective value —14 solves the 


Il 


original problem. 


Economic Interpretation 


The decomposition algorithm has an interesting economic interpretation. Con- 
sider the case of a large system that is composed of smaller subsystems indexed 
by i= 1,...,7. Each subsystem i has its own objective, and the objective function 
of the overall system is the sum of the objective functions of the subsystems. 
Each subsystem / has its constraints designated by the set _X,, which is assumed 


to be bounded for the purpose of simplicity. In addition, all the subsystems share 
a few common resources, and hence, the consumption of these resources by all 
the subsystems must not exceed the availability given by the vector b. 
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Recalling the economic interpretation of the dual variables (Lagrangian 
multipliers), w, is the rate of change of the objective as a function of ;. That is, 


if b; is (marginally) replaced by 5; + A, holding the current nonbasic variables in 
the master problem at zero, then the objective value is modified by adding w,A. 
Hence, —w, can be thought of as the price of consuming one unit of the ith 
common resource. Similarly, —@; can be thought of as the price of consuming a 


portion of the ith convexity constraint. 

With this in mind, the decomposition algorithm can be interpreted as 
follows. With the current proposals of the subsystems, the superordinate (total 
system) obtains a set of optimal weights for these proposals and announces a set 
of prices for using the common resources. These prices are passed down to the 
subsystems, which modify their proposals according to these new prices. A 
typical subsystem i solves the following subproblem: 

Maximize (wA; —¢;)x; +@; 
subject to x; € X;, 


or equivalently, 

Minimize (¢; ~WwA,; )x; —a; 

subject to x; € X;. 
The original objective function of subsystem i is ¢;x;. The term —wA;x; reflects the 
indirect price of using the common resources. Note that A;x, is the amount of 
the common resources consumed by the proposal x; . Since the price of using 
these resources is —w, then the indirect cost of using them is -wA;x,, and the 


total cost is (¢; — wA;)x;. Note that the term —wA;x; makes proposals that use 


much of the common resources unattractive from a cost point of view. Subsys- 
tem / announces an optimal proposal x,,. If this proposal is to be considered, 


then the weight of the older x;;—proposals must decrease in order to “make 
room” for this proposal; that is, ) ;4,; must decrease from its present level of 1. 


The resulting saving is precisely @;. If the cost of introducing the proposal xj, 
is less than the saving realized; that is, if (e;-WA,)x;,—-a@;<0, or 
(wA,; —¢;)x;, +@; >0, then the superordinate would consider this new pro- 


posal. After all the subsystems introduce their new proposals, the superordinate 
determines an optimal mix of these proposals and passes down new prices. The 
process is repeated until none of the subsystems has a new attractive proposal; 
that is, when (c; —-wA;)x; —@; 2 0 for each i. 
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7.6 DUALITY AND RELATIONSHIPS WITH OTHER 
DECOMPOSITION PROCEDURES 


The Dantzig—Wolfe decomposition procedure discussed in this chapter is 
equivalent for linear programming problems to two other well-known partition- 
ing/decomposition/relaxation techniques, namely the Benders decomposition 
and the Lagrangian relaxation methods To expose this relationship, consider a 
linear programming problem P of the following form. (The choice of equalities 
and inequalities is only for the sake of illustration; other forms may be readily 
handled.) 


P: Minimize ex 
subjectto Ax=b 
xe X ={x: Dx 2d,x 20}, 


where A is m Xn and D is m’ xX n. For convenience, let us assume that X is 
nonempty and bounded. (We ask the reader in Exercise 7.36 to explore the 
following connections for the case of unbounded X.) 

Now, let us write the dual D to Problem P. We designate w and v as the 
dual variables associated with the constraints Ax = b and Dx > d, respectively. 


D: Maximize wb+vd 
subjectto wA+vD<c 
w_ unrestricted, v > 0. 


Observe that when w is fixed at some arbitrary value, we obtain a linear pro- 
gramming problem in the variables v. In particular, this linear program may be 
specially structured or easy to solve. Assuming this to be our motivation, let us 
proceed by partitioning Problem D, while treating the variables w as complicat- 
ing variables as follows: 


wb + maximum vd 


D: Maximize subject to vD <e-—wA 
WwW unres 
v20 
= Maximum ~ + minimum (¢- wae! (7.13) 
Ww ounres xe X 


Here, we have written the dual to the inner optimization problemat the last step, 
noting the definition of X. Since X is assumed to be nonempty and bounded, the 
inner minimization problem in Equation (7.13) attains an extreme point optimal 
solution. Denoting xj,...,x, as the vertices of X as before, we have that D is 


equivalent to the problem of maximizing {wb + minimum ;-1,,_ ¢(¢— WA)x ;} 


over unrestricted values of w. Denoting z as the objective function in {-}, this 
may be rewritten as the following master problem: 
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MP: Maximize z 
subjectto z< wh + (c—wA)x ; for j=1,..,¢ (7.14) 


Z,W unrestricted. 


Note that MP is inconvenient to solve directly because it typically has far too 
many constraints. Hence, we can adopt a relaxation strategy, in which only a 
few of the constraints in Equation (7.14) are explicitly maintained. Suppose that 
for such a relaxed master program we obtain an optimal solution (Z,w). Then Z 


is an upper bound on the optimal value to the original problem (why?). Further- 
more, (Z,W) is optimal for MP if and only if (Z,W) is feasible to all constraints 


in Equation (7.14) (why?). Hence, in order to check if any constraint in Equation 
(7.14) is violated, we wish to check if 7 < wb + (c— wA)x; for all j= 1....,¢, that 
is, if 7 << wb+ minimum ;_;,_,{(¢- WA)x ;}. However, the latter problem is 


equivalent to the linear programming subproblem: 


wb + minimum {(c — WA)x}. (7.15) 
xeX 


If Z is less than or equal to the optimal objective value in Equation (7.15), then 
we are done. (In fact, in this case, Z will be equal to the optimal value in Equa- 


tion (7.15) since Z is equal to Wh+(c-WA)x, for some j€ {l,...,¢} in the 
solution to the relaxed master program.) Otherwise, if x, solves Problem (7.15), 
we have Z>Wb+ (c—WA)x;, and we can generate the constraint 
z<wb+(c—wA)x;,, and add it to the current (relaxed) master program and 


reoptimize. Observe that this new constraint cuts off or deletes the previous 
master problem solution (Z,w). This process may be repeated until the solution 


(Z,wW) to some relaxed master problem gives Z equal to the optimal value in 


Equation (7.15). This must occur finitely since X has only a finite number of 
vertices. 

The foregoing procedure is known as Benders partitioning or Benders 
decomposition technique. (Its generalization to nonlinear or discrete problem 
structures is addressed in Exercise 7.38 and in the Notes and References 
section.) Problem (7.14) (or its relaxation) is referred to as Benders (relaxed) 
master problem, its constraints are referred to as Benders constraints or Benders 
cuts (from their role of deleting previous master problem solutions), and 
Problem (7.15) is known as Benders subproblem. However, note that Problem 
(7.15) is also the subproblem solved by the Dantzig-Wolfe decomposition 
method, and that the Benders master program (7.14) is simply the dual to the 
Dantzig—Wolfe master problem with Constraints (7.1)}(7.3). (Here, the 
nonnegative dual multipliers associated with the constraints (7.14) are A ore hs 


1,...,t; these variables sum to unity by virtue of the column of the variable z in 
Problem MP.) Therefore, Benders algorithm is called a row generation 
technique in contrast with the Dantzig—Wolfe column generation procedure. It 
follows that if at the end of each relaxed master program solution, we maintain 
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only those generated rows in Equation (7.14) that have currently nonbasic slack 
variables, and so whose complementary dual variables Aj are basic, then the 


resulting algorithm is precisely the same as the decomposition algorithm of 
Section 7.1. In particular, the starting solution technique of Section 7.3 provides 
an initial bounded Benders master program. Also, when Benders algorithm 
terminates, the available optimal primal solution may be computed as the sum of 
the products of the x ;-solutions and the corresponding dual variables Aj that are 


associated with the explicitly present constraints (7.14) in the final Benders 
relaxed master problem. (Note that this sum is a convex combination.) 
Furthermore, note that if we did not delete any of the constraints generated by 
Benders algorithm and solved each relaxed Benders master program to 
optimality, the equivalent implementation for the Dantzig-Wolfe method would 
be to optimize each master program over all the Aj; -variable columns generated 


thus far. 
Observe that the foregoing discussion also indicates how to recover an 


optimal set of dual multipliers (w*,v") for the Problem P after solving it using 


the Dantzig—Wolfe decomposition algorithm. The vector w’ is directly available 
as the optimal set of multipliers associated with the constraints (7.1). The corre- 


sponding solution v* is obtainable from the first inner maximization problem in 
Equation (7.13) after fixing w at w’. Hence, if the second inner minimization 


subproblem in Equation (7.13) is solved with w = w*, then v” is obtained as the 
set of optimal dual multipliers associated with the constraints in X. For instance, 


in the example of Section 7.2, we obtain (w,, W>) = (-1, ~1). Furthermore, with 
w = w’ fixed, the (final) subproblem solved is to maximize {-3x,:x€ X}, 


which yields an optimal set of dual multipliers v" = (0, 0, 0, 0) for the 
constraints 3, 4, 5, and 6 in the example. 

It may also be noted that the Dantzig—Wolfe and the Benders decompo- 
sition methods are respectively classified as price directive and resource 
directive schemes. The former type of procedure conducts the coordination in 
the decomposition method by adjusting prices or objective function coefficients 
via Lagrangian multipliers. On the other hand, the latter type of procedure is 
dual to this scheme, and conducts this coordination by adjusting the common 
resource availabilities (right-hand-sides) by fixing certain variable values. 

There is another general price—directive optimization strategy for P that 
finds an equivalence with the foregoing methods when solving linear program- 
ming problems, namely the Lagrangian relaxation technique (See Exercise 
6.71.) Observe from Equation (7.13) that if we denote the function in {-}, which 
is a function of w, as @(w), we can equivalently state D as the following 


problem: 


Maximize {@(w) : w unrestricted} (7.16) 
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where 
O(w) = wb+minimum {(c—wA)x:x eX}. (7.17) 


Problem (7.16) is known as a Lagrangian dual to Problem P. Problem (7.17) is 
the associated Lagrangian subproblem. Note that in Problem (7.17), the con- 
straints Ax = b have been accommodated into the objective function of P via the 
Lagrangian multipliers w. By the foregoing discussion, it is evident that the 
maximum value in Problem (7.16) matches with the optimal objective value for 
P, and hence, @(w) provides a lower bound for this optimal value for any 


arbitrary w. This is why Problem (7.16) is called a “Lagrangian dual” problem. 
Furthermore, since 6(w) = wb + minimum ;_)_,{(¢-WA)x;}, we have that 


O(w) is a piecewise linear and concave function of w (why?). Hence, Problem 
(7.16) seeks an unconstrained maximum for this function. Although several 
nonlinear search methods for nonsmooth optimization problems may be used to 
solve Problem (7.16), the Benders procedure is evidently one viable solution 
technique. This cutting plane or row generation scheme can be viewed as a 
tangential approximation method for solving Problem (7.16), in which only a 
few of the tangential supports or segments z < wb + (¢—WA)x ;, / = 1,...,¢ that 


describe 9( -) are generated. Here, z is the value measured along the O( -)-axis. 
Using this viewpoint, the relaxed master problem MP from Equation (7.14) 
employs an outer linearization of Problem (7.16), using only a subset of the 
tangential supports that actually describe the piecewise linear function 6(-). If W 
solves this problem with objective value Z, then Z is an upper bound on the 
optimal value of Problem (7.16). However, if Z=O(W) itself, then W_ is 
optimal for Problem (7.16) (why?). Otherwise, Z>O(W)= Wb+ 
minimum {(¢ — WA)x : x e X} = Wb +(c—WA)x,, say, as produced by the Sub- 
problem (7.17), which coincides with the Benders subproblem (7.15). Hence, 
the tangential support z < wb+(c—WwA)x, is generated and included in the 
outer-linearization of @(-), and the procedure is repeated until termination is 
obtained. At termination, the optimal primal solution may be recovered by 
weighting each x ; in Equation (7.14) by the corresponding dual multiplier 4,, if 
positive at optimality, and summing. 

Exercises 7.42 and 7.43 explore further refinements to accelerate the 
convergence behavior of Benders and Lagrangian relaxation approaches using 
the boxstep method and its extensions, as well as the associated corresponding 


stabilized column generation implementation techniques for the Dantzig-Wolfe 
decomposition procedure. 


Example 7.3 


Consider the linear programming problem given in the example of Section 7.2. 
Denote the dual variables associated with the constraints as (w,,W2,¥,V2,V3, 


v4) and write the dual as follows: 
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Maximize 2w, + 3w, + 2H, + Svy + 293 + 64 


subjectto weo+ Wy + ww + Ww BS ed. 
W> + 25 < -l 
Ww; - vw + 24 < -l 
2W + 3+ wy SS 1 
w<0,v<0 


Treating w as the complicating variables, we obtain the Benders master problem 
in this case as follows: 


Maximize Z 
subjectto zS2w, +3w) +xj(-2— m4 — w) 
+Xj2(—1— wy) + xj3(-1- 4) + xj4(1—-2 my) 
for j =1,...,¢ 
Zz unrestricted, w < 0, 
where Xj = (Xj1,%j2. X73 Xa). = ],..., t, are the vertices of X as defined in 
Section (7.2). The corresponding Lagrangian dual problem is given as follows: 
Maximize {@(w):w <0} 
where, for a given w = (w,, >), we have 
O(w) = 2m, +3 + minimum{(—2 — wy, — w2)xy + (-1— Wy) 2x5 
xeX 
+ (-1— 9 )2xy + (1-2 Wy xy}. 


Note that evaluating @(w), given w, is precisely equivalent to solving the 


Benders subproblem. 
Suppose that as in Section 7.2 we begin with the vertex x, = (0, 0, 0, 0) 


of X. Hence, the relaxed Benders master problem is of the form: 
Maximize Zz 


subjectto z<2w,+3w, 
w sO. 


The optimal solution is 7 =0 and w =(0, 0). Hence, using only the tangential 
support z< 2w,+3w, for 6(-), we obtain w = (0, 0) as the maximizing solu- 
tion. Solving the Benders subproblem with w = (0, 0), that is, computing 0(W), 
we get @(w) =—17/2, which is realized at x» =(2,3/2,3,0). Since 0(W) <Z, 
we generate a second Benders cut or tangential support using x5: 
ZS 2w, +3w, + 2(-2—w, — Ww) + (3/2) (-1- wy) + 3(-1- wy) 
=-17/2—3w, — (1/2). 


This leads to the following relaxed Benders master program: 
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Maximize z 

subject to z<2w +3w, 
z$-17/2-3w, -(1/2)w» 
w <0. 


The optimal solution is Z = -—17/2 and w = (17/10, 0). The slack variables in 
both constraints are nonbasic, and hence, both constraints are retained. (The 
respective dual variables are 3/5 and 2/5.) Solving the Benders subproblem, we 
compute @(w) =—59/10, which is realized at x3; = (0, 5/2, 0, 0). Since 


Z > O(w), we generate a third Benders constraint or tangential support as 
ZS 2w, +3w, + (5/2)(-1— wp) = —5/2+ 2w, + (1/2). 
Appending this to the master problem, we obtain: 


Maximize z 

subjectto z<2w +3w 
z$-17/2-3w -(1/2)w, 
z<$-5/2+2w, +(1/2) w» 
w <0. 


The optimal solution to this problem is Z = —49/10 and w = (-6/5, 0), with the 
slack in the first constraint being basic. (The optimal dual multipliers are 2, = 0, 
Ay = 2/5, and A; = 3/5.) Hence, the first constraint may be deleted from the 


current outer—linearization. (A deleted constraint can possibly be regenerated 
later.) Observe how these computations relate to the calculations in the 
numerical example of Section 7.2. We ask the reader in Exercise 7.37 to 
continue the solution process by next computing 6(w) and, finally, to recover 


primal and dual optimal solutions upon completion. 
EXERCISES 


[7.1] Use the Dantzig—Wolfe decomposition principle to solve the following 
problem: 


Maximize 3x, + Sx, + 2x3; + 3x4 
subject to 2x, + 4x, + 5x3 + 2x4 S$ 7 
2x] + 3X, < 6 
x + 4x» < 4 
3x3 + 4x4 2 12 
x3 < 4 
X4 < 3 
xX, Xp, x3, xy 2 0 


[7.2] Solve the following problem by the Dantzig—Wolfe decomposition 
technique: 
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Minimize —x, 


subjectto x, + 2x, + 3x3 + x4 2 40 
x1 + Xz < 2 

—X) + 2x5 < 2 

X3 + X4 = 8 

xX, Xp, X3, x4 2 0. 


[7.3] Solve the following problem by the Dantzig—Wolfe decomposition 
algorithm: 


Minimize —x, — 2x, —- 3x3 — 2x4 
subjectto 3x, + XxX, + 2x3 + XxX S< 12 
-X) + Xp < 4 
—X, + 2x5 < 12 
3x3 + 2X4 < 9 
X15 X95 X3, X4 > 0. 


[7.4] Consider the following linear programming problem: 


Maximize 
subject to x, +3x> <9/4 
2x; — 3x <0 
xe X ={(x,X)):05 x S108 x <I}. 


a. Solve this problem graphically. 

b. Solve using the Dantzig—Wolfe decomposition method. Comment on 
the number of extreme points of X used at optimality in terms of 
Remark 6 of Section 7.1 

c. Does the Dantzig—Wolfe master program (7.1)-(7.3) have alternative 
optimal solutions? Does the original problem have alternative opti- 
mal solutions? Explain. 


[7.5] Solve the following linear programming problem by the Dantzig—Wolfe 


decomposition method using one convexity constraint. Show the progress of the 
lower bound and primal objective. Obtain primal and dual solutions. 


Minimize —x,} — 3x, + %3 - % 
subjectto x) + XxX, + %3 + XX S 7 
x} + x2 < 5 
x3 “bh 2X4 < 6 
— xX, + X% < 3 
X15 X25 x3; X4 > 0. 


[7.6] Solve the following problem using the Dantzig—Wolfe decomposition 
technique with one convexity constraint. Show the progress of the lower bound 
and primal objective value. Obtain primal and dual solutions. 
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Minimize -x, - 2x, - 2x3 - Xq 
subjectto x + 2x, + 3x3 + x4 < 40 
—X) + X. + 2x, + x4 < 10 
xX + 3x, < 30 
2x, + XX < 20 
x3 < 10 
X4 < 10 
x3 =F X4 < 15 
x, xX), X3, xy 2 0. 


[7.7] Solve the following problem by the Dantzig—Wolfe decomposition 
technique using two convexity constraints: 


Maximize 3x) + xX, + 3x3 -— X4 
subjectto 2x) + xX» + x3 + x4 < 12 
—X) + X5 < 2 
3x — 4x3 < 5 
x3 +x, < 4 
= X3 + X4 < 3 
X> X92, x3; X4 > 0. 
[7.8] Consider the following problem: 
Minimize x — 3x2 - 4x3 
subjectto 3x, + 2x, + 1x3 < 6 
X + =X -— 2x3 2 2 
0< x, X, x3 < 3. 


a. Set up the problem so that it can be solved by the Dantzig—Wolfe 
decomposition algorithm. 
Find a starting basis in the A-space. 

c. Find an optimal solution by the Dantzig—-Wolfe decomposition 
algorithm and compare this approach with the bounded variables 
simplex method. 


[7.9] Apply the Dantzig—Wolfe decomposition algorithm to the following 
problem: 


Minimize -3x,) + Xx, —- 5x3 
subjectto 6x, -— 2x, + 3x3 < 4 
O<%x, xX, X3 <i. 


[7.10] Consider the following (transportation) problem: 
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Minimize Xyyo+ 5X12 + 4x13 + 3x14 + 2X9] + 3X99 + 4x53 + 5X4 
subject to Ny oF M2 + HZ + Xyq = 600 
XQ) + X20 + X23 + X44 = 800 
x11 + X91 = 300 
X12 + X99 = 300 
13 + X93 = 400 
X14 + X24 = 400 
Mir X25 M3 Mg» X21, 22> 03, XQ 2. 


a. Set up the problem so that it can be solved by the Dantzig—Wolfe 
decomposition algorithm using four convexity constraints. 
b. Find an optimal solution using this decomposition algorithm. 


[7.11] Solve the following linear program entirely graphically using Dantzig— 
Wolfe decomposition: 


Minimize 3x, + 5x. + 3x, - 2x4 + 3x5 

subjectto x + XxX + % + XxX > 3 
3x + x2 + 5x3 + x4 7 2X5 > 6 
xy + 2x3 — X4 > 2 
x, x7, x3, X4, Xs > 0 


(Hint: Let the first constraint denote Ax > b and the next two constraints 
represent X. Then take the dual of each set.) 

Indicate how this approach may be generalized to any number m of, say, 
equality constraints. (Hint: Let the first constraint denote Ax = b and the 
remaining m — | constraints be part of the subproblem. Then treat the 
subproblem in a similar way as above.) 


[7.12] Construct both a flow chart and detailed steps of the Dantzig—Wolfe 
decomposition algorithm for solving the problem: Minimize ex subject to Ax = 
b, x e X, where X is not necessarily bounded. 


[7.13] Is it possible that the Dantzig—Wolfe decomposition algorithm would 
generate an optimal nonextreme point of the overall problem in case of 
alternative optimal solutions? Discuss. (Hint: Consider the following problem 
and start with the extreme point (0, 0): 


Maximize Xx + XH 
subject to Mm + xX S 3/2 
O<x, 2) < 1.) 


[7.14] Solve the following problem by the Dantzig—Wolfe decomposition 
algorithm. Use Phase I to get started in the A-space. 


Maximize 4x, + 3x27 + x3 
subjectto x + X% + % < 6 
3x + 2X => 6 
—X + XQ = 2 
2X3 > 5 
xX}, X75 x3 > 0 
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{7.15} An agricultural mill produces cattle feed and chicken feed. These prod- 
ucts are composed of three main ingredients, namely, corn, lime, and fish meal. 
The ingredients contain two main types of nutrients, namely, protein and cal- 
cium. The following table gives the nutrients’ contents in standard units per 
pound of each ingredient: 


INGREDIENT 
NUTRIENT CORN LIME _ FISH MEAL 
Protein 27 20 30 
Calcium 20 30 20 


The protein content must lie in the interval [22, 25] per pound of cattle feed. 
Also the calcium content must be greater than or equal to 22 per pound of the 
cattle feed. Similarly, the protein content and the calcium content must be in the 
intervals [21, 24] and [21, 29], respectively, per pound of the chicken feed. Sup- 
pose that 3000, 2500, and 1000 pounds of corn, lime, and fish meal, 
respectively, are available. Also, suppose that it is required to produce 4000 and 
2000 pounds of the cattle and chicken feed, respectively. Let the price per pound 
of the corn, lime, and fish meal be respectively $0.20, $0.15, and $0.25. 
Formulate the blending problem with an objective of minimizing the cost. 

Solve the problem by the Dantzig—Wolfe decomposition algorithm using 
two convexity constraints. Extra corn and fish meal can be obtained but, because 
of shortages, at the higher prices of $0.22 and $0.27 per pound. Would you 
advise the mill to consider extra corn and fish meal and modify their blending at 
these prices? Why or why not? 


{7.16] A company has two manufacturing facilities, one in Atlanta and one in 
Los Angeles. The two facilities produce refrigerators and washer/dryers. The 
production capacities of these items in Atlanta are 5000 and 7000, respectively. 
Similarly, the capacity of the Los Angeles facility is 8000 refrigerators and 4000 
washer/dryers. The company delivers these products to three major customers in 
New York City, Seattle, and Miami. The customer’s demand is given below: 


DEMAND / CUSTOMER _ NEW YORK SEATTLE MIAMI 
Refrigerators 4000 5000 4000 
Washer/dryers 3000 3000 4000 


The items are transported from the manufacturing facilities to the customers via 
a railroad network. The unit transportation costs (no distinction is made between 
the two items) are summarized below. Also, because of limited space, the 
maximum number of refrigerators and/or washer/dryers that can be transported 
from a facility to a customer is given in the following table: 


CUSTOMER 
FACILITY NEW YORK SEATTLE MIAMI 
pore Unit shipping cost $ 7 15 8 
Max. number of units 6000 3000 8000 
Unit Shipping cost $ 15 12 20 


Los Angeles Max. number of units 3000 9000 3000 
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It is desired to find the shipping pattern that minimizes the total transportation 
cost. 
a. Formulate the problem. 
b. Use the Dantzig—Wolfe decomposition technique with two convexity 
constraints to solve the problem. 


(Note: This problem is called a multicommodity transportation problem. The 
subproblem decomposes into two transportation problems. If you are familiar 
with the transportation algorithm, you can use it to solve the subproblems. 
Otherwise, use the simplex method to solve the subproblems.) 


[7.17] A company owns two refineries in Dallas and New York. The company 
can purchase two types of crude oil: light crude oil, and heavy crude oil at the 
prices of $15 and $10 per barrel, respectively. Because of shortages, the maxi- 
mum amounts of these crudes that can be purchased are 3 million and 2 million 
barrels, respectively. The following quantities of gasoline, kerosene, and jet fuel 
are produced per barrel of each type of oil: 


GASOLINE KEROSENE JET FUEL 


Light crude oil 0.40 0.20 0.35 
Heavy crude oil 0.32 0.40 0.20 


Note that 5 percent and 8 percent of the light and heavy crude are lost during the 
refining process, respectively. The company has contracted to deliver these 
products to three consumers in Kansas City, Los Angeles, and Detroit. The 
demands of these products are given below: 


GASOLINE KEROSENE JET FUEL 


Kansas City 300,000 800,000 - 
Los Angeles 600,000 400,000 800,000 
Detroit 900,000 300,000 500,000 


It is desired to find the amounts that must be purchased by the company of each 
crude type at each of its refining facilities, and the shipping pattern of the 
products to Kansas City, Los Angeles, and Detroit that satisfy the demands and 
minimize the total cost (purchase plus shipping). The shipping and handling of a 
barrel of any finished product from the refineries to the consumers is given 
below: 


KANSAS CITY _LOS ANGELES _ DETROIT 
Dallas Refinery $0.70 $0.50 $0.70 
New York Refinery $0.40 $0.90 $0.40 


a. Formulate the problem. 
b. Suggest a decomposition scheme for solving the problem. 
c. Solve the problem using your scheme in Part (b). 


[7.18] Assume that a linear program requires 3m/2 iterations for a solution. 
Also, assume that standard techniques of pivoting are used to update the basis 
inverse and RHS vector [together, these constitute an (m + 1) x (m+ 1) matrix if 
we ignore the z column]. If there is no special structure to the constraint matrix, 
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then is there an optimal split for Dantzig—Wolfe decomposition? That is, find 
m, + m, =m such that the first m, constraints form the master problem and the 
next m constraints are subproblem constraints, and the total “effort” is 
minimized. Let the “effort” be defined by the number of elementary operations 
(additions, subtractions, multiplications, and divisions). 

{7.19] In the previous problem suppose that , and mp are given and that the 
second mp constraints are of a special structure. Specifically, suppose that the 


subproblem requires only 5 percent of the normal effort to yield a solution when 
treated by itself. 

a. Should the problem be decomposed for efficiency? 

b. Is there a critical value of the percentage effort required? 


[7.20] Develop the master and the subproblem using the Dantzig—Wolfe 
decomposition technique for the following linear programming problem. Assume 
that X is polyhedral and has a special structure. Formally state the decomposition 
algorithm for this case. 


Maximize ¢x+dy 
subjectto Ax+Dy<b 
xeX, 


[7.21] Consider the problem: Minimize ex subject to Ax = b, Dx = d, x > 0. 
Suppose that w; and w> are the optimal dual solution vectors associated with 
the constraints Ax = b and Dx = d, respectively. Consider the problem: 
Maximize (w;A~c)x subject to x eX ={x:Dx =d,x 20}. Assume that X is 
bounded. Let x;,x>,...,x; be alternative optimal solutions of the foregoing 


problem that are extreme points of the set X. Show that an optimal solution of 
the original problem can be represented as a convex combination of 


* * * . 
X1,X9,-..X;, that Is, 


SIMs iT M> 
a 
i 


IV 
oO 


JHlwok. 


Is this true if X is unbounded? Prove this assertion or give a counterexample and 
modify the representation statement. 


{7.22] Consider the feasible and bounded problem: Minimize ex subject to Ax 
= b, x ¢ XY. Suppose that w"A—c=0 where w’ is the optimal Lagrange multi- 
plier vector associated with the constraints Ax = b. Does this imply that there 


exists an optimal solution x” that belongs to the interior of X? Is the converse of 
this implication true? Interpret your answers geometrically. 
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[7.23] Consider the problem: Minimize ex subject to Ax = b, x e X. Suppose 
that X has a block diagonal structure. The Dantzig—Wolfe decomposition 
algorithm can be applied by using either one convexity constraint or several 
convexity constraints, one for each subproblem. Discuss the advantages and 
disadvantages of both strategies. Which one would you prefer and why? 


[7.24] Develop the derivation of a lower bound on the optimal objective value 
when using the Dantzig—Wolfe approach for the case of an unbounded sub- 
problem region. 


[7.25] Suppose that the columns added during each master step of the Dantzig— 
Wolfe decomposition algorithm are stored. In particular, suppose that the master 
problem at iteration p is as follows: 


P 
Minimize > (ex;)A; 
j=l 


P 
subjectto  (Ax;)A; =b 
j=l 


P 
XA; =1 
j=l 
Aj20, falas Ds 


where Xj,...,X, are the extreme points generated so far. Discuss the details of 


P 
such a decomposition procedure and compare this with that of Section 7.1, 
Illustrate by solving the problem of Section 7.2. 


[7.26] Referring to Exercise 7.25, consider the following master problem: 
Pein P 
Minimize > (¢x;)A; 
j=l 


P 
subjectto > (Ax;)A; =b 
j=l 
P 
j=l 
Aj >0, j=l,...5D- 


Write the dual of this master problem. Suppose that the dual problem has been 
solved instead of this problem; how does the decomposition algorithm proceed? 
Give a detailed algorithmic statement and relate to the discussion in Section 7.6. 
Show convergence of the procedure and interpret it geometrically. 


[7.27] Give a detailed analysis of the cases that may be encountered as artificial 
variables are used to find a starting basis of the master problem in Dantzig— 
Wolfe decomposition. Discuss both the two-phase method and the big-M 
method. 
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{7.28] Consider the following cutting stock problem. We have standard rolls of 
length £, and an order is placed requiring 5; units of length £;, where i= 1.,..., 


m. It is desired to find the minimum number of rolls that satisfy the order. 


a. Formulate the problem. 
b. Apply the Dantzig—Wolfe decomposition algorithm to solve the 
problem, neglecting integrality requirements. Discuss in detail. (Hint: 


Consider the column a ; representing the jth cutting pattern. Here, a; 


J 


is a vector of nonnegative integers; a;;, the ith component, is the 


number of rolls of length £; in the jth cutting pattern. Develop a 


scheme for generating these a 


problem and the subproblem?) 


[7.29] Consider the following problem: 


Minimize ¢oXxo 
subject to Doxo 
D)Xo 


D Xx 


Drx9 
Xo r 


+ €)Xy 
+ A)Xx) 
+ B,x, 


X]> 


J 
+ €9X0 + 
+ A»X> + 
+ Bx) 
X), 


-—columns. 


s+ Cr 
a Ar 


+ Br 


“5 


XT 
Xr 


Xr 
XT 


2 


What is the master 


Describe in detail how the Dantzig—Wolfe decomposition technique can be used 
to solve problems of the foregoing structure. (Hint: Let the first set of 
constraints be the constraints of the master problem. The subproblem consists of 
the remaining constraints. Take the dual of the subproblem and solve it by 
decomposition. This becomes a “three-level” algorithm. The subproblem is said 
to have a dual—angular structure.) 


[7.30] Consider the following staircase structured linear program: 


Minimize ¢)X; + €)X, + 3X3 + €4Xgt---+ 7X7 


B5x> +: A3X3 


subject to 
AjX| 
B, x] + Axx) 
X]; X2; 


B3x3 + A4Xx4 


X3, 


X4; 


=b 
=b, 
=b; 
=b, 


Br_|X7_) + Apx7 =br 


XT 


-l, 


xr > 0. 


a. Writing x; as a convex combination (with weights 4, pt ahi Bi) 


of vertices in X) = {x: A,x, =b),x, 20}, assumed to be nonempty 


and bounded, use the Dantzig—Wolfe decomposition principle on the 


problem. 
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b. For the master program obtained in Part (a), define X> as the set of 
A,j, J =1,...,£, and x that satisfy the constraints involving only 


these variables (that is, not involving the variables x3,...,x7. Use the 
Dantzig—Wolfe decomposition principle on this master problem with 
subproblems solved over X to obtain a second-level nested 
decomposition of the problem. 


c. Continuing in this fashion, propose a multilevel nested decomposi- 
tion algorithm for this problem. 


{7.31] Solve the following generalized linear programming problem by 
decomposition: 


Minimize -2x, + 5x, - 4%; 
subject to xX + 2x, + ax < 6 
3x; = 6x + AnX3 < 3 
3a, + 2a = 6 
X15%25X3,4,a, 2 0. 
[Hint: Let X = {(a,,a7):3a, +2ay = 6, a,ay = 0}.] 
[7.32] Consider the following problem: 
. . . T 
Minimize > ¢;x; 
i=l 
: T 
subject to > A;x;=b 
i=l 
x; € X;, ico) Vee (8 


Show that any basis in the A-space for the Dantzig—Wolfe master program 
having 7 convexity constraints must contain at least one Ay for each i = 1,...,7. 


[7.33] Many options are available while solving the subproblem(s) and the 
master problem in Dantzig—Wolfe decomposition. These include the following: 
a. The subproblem is terminated if an extreme point x, is found with 


Zp —Cy > 0. Then A, is introduced in the master problem. 


b. Several columns can be generated from the subproblem(s) at each 
iteration. 

c. At least one additional column is added to the master problem while 
explicitly storing all the previously generated columns. In this case 
the master problem reduces to finding an optimal mix of all columns 
generated so far. 


Discuss in detail the foregoing options and compare and contrast them. 
Elaborate on the advantages and disadvantages of each option. 


[7.34] Consider the following problem: 
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. . . T T 
Minimize 2De;x;+ Vy; 
j=l j=l 
subject to x ;_ —x,+A,y; =b;, J=1,..4T 
XT =b 
O<x; Su,;, =161 
O<y,; <u}, =1).,2 


where xp is a known vector. 


What class of problems lend themselves to this general structure? 


What is the interpretation of the vectors x; and y ;? (Hint: Examine 


How can the Dantzig—Wolfe decomposition algorithm be applied to 


this system? (Hint: Choose every other constraint to form the master 


a 


Apply the procedure in Part (b) to solve the following problem, using 


a. 
a discrete control system.) 
b. 
constraints. ) 
c. 
Xo = 80: 
Minimize Xt X_ +%3+%X4 +29, + 5y> +493 +694 
subject to x9 — x + y 
x — XQ 29 
X2 — *3 
Xz -—X4 
*4 
[7.35] Consider the following problem: 
Minimize ¢)x; + ¢9x.+ 
subject to Ax] + A»X> + 
B)x, 
Box) 
X]; X25 aang 


= 40 
= 50 
+3 = 60 
+V4 = 40 
= 30 
OS X,X2, %3,X4 < 40 
O< W.¥2 < 40 
0S y3,¥4 $50. 
CrXr 
Arxr < b 
< b 
< bo 
Brxr << br 
Xr 2 0. 


The following implementation of the Dantzig—Wolfe decomposition principle is 


a possibility. The subproblem constraints are: 


AX; 
X]> 


+ Adxo+ 
X25 


“5 


ons oe Ar 


Xr 
Xr 


IV IA 
= 
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and the master constraints are B)x, <b,, B>x> < bo,...,B7x7 <b. Describe the 


details of such an algorithm. What are the advantages and the disadvantages of 
this procedure? Does this scheme have an economic interpretation? Use the 
devised algorithm to solve Exercise 7.2. 


[7.36] Provide the development of Benders partitioning algorithm and the 
Lagrangian relaxation method given in Section 7.6 for the case of unbounded X 
(without artificially bounding this set). Give a formal and complete statement of 
the algorithm you develop and establish its convergence. Illustrate by applying 
this algorithm to Example 7.1, and identify the step-by-step relationship of this 
procedure with the Dantzig-Wolfe decomposition method of Section 7.4. 


[7.37] Complete the solution of Example 7.3 in Section 7.6, and interpret the 
solution process as an application of the Benders and the Lagrangian relaxation 
techniques. Provide both primal and dual solutions. 

[7.38] Consider the generalization of Benders partitioning algorithm for the 
following nonlinear and/or discrete problem: 


Minimize ex + f(y) 
subjectto Ax + By=b 
x 20 
y eY, 


where c, b, x, y are vectors, A, B are matrices, fis an arbitrary function, and Y is 
an arbitrary set. 
a. Show that the problem can be reformulated as follows: 
Minimize z 
subject to z> f(y)+w(b-By) for each we W 
yey, 


where W = {w unrestricted: wA < c}. (Hint: The original problem can be 
reformulated as follows: 


Minimizey f(y) + minimum ex >. 
yeY Ax=b-By 
x>0 


Now, take the dual of: 
Minimize ex 


subject to Ax =b- By 
x 20.) 


b. Show that z > f(y) + w(b - By) for each wA < ¢c if and only ifz > fly) 
+ w,(b — By) and d;(b — By) < 0, for each extreme point w; and 
every extreme direction d ;, respectively, of the region W. 


c. Make use of Part (b) to reformulate the problem in Part (a) as 
follows: 
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Minimize z 
subjectto z2 f{y)+w,(b-By), j=1,....t 
d ;(b-By) <0, J=),...52@ 
yey, 
where Wj,...,W, and dj,...,d, are, respectively, the extreme points 
and the extreme directions of W. 
Without explicitly enumerating wy,...,W, and d),...,d, beforehand, 
devise a decomposition algorithm for solving the problem in Part (c). 
(Hint: Devise the Master Problem (MP) and Subproblem (SP) as 
below: 
MP: Minimize z 
subjectto z2f(y)+w,(b-By), j= ee 
d ;(b-By) <0, j=... 0 
yey, 
where wj,....W, and dj,...,dy are, respectively, the extreme points 
and extreme directions generated so far. 


SP: Maximize w(b-By) 
subject to wa <c, 


where y is obtained from the optimal solution to the master problem.) 
How would you obtain the optimal (x, y) at termination of the 
decomposition algorithm in Part (d)? (Note: This algorithm is 
Benders partitioning procedure. Note that the set Y can be discrete, 
and so, the procedure can be used for solving mixed-integer 
problems. In this case, the master problem in Part (d) is a pure 
integer programming problem and the subproblem is a linear 
program.) 


[7.39] Apply Benders partitioning procedure of Exercise 7.38 to solve the 
following problem [let ((x,,x2) be x and (x3,x4) bey]: 


Minimize -—x, — 2x. - 3x; - X%%4 
subjectto x + 2x9 + 2x3 + x4 S$ 12 
—3x + 2X9 < 6 
2x4 a x2 < 6 
3x3 + X4 < 8 
X5 X, x3, xy 2 0. 


[7.40] A company is planning to build several warehouses for storing a certain 
product. These warehouses would serve two major customers having monthly 
demands of 3000 and 5000 units. Three candidate warehouses having respective 
capacities 4000, 5000, and 6000 can be constructed. Using the estimated 
construction cost of the warehouses, their useful life, and time value of money, 
the construction cost per month for the three warehouses is estimated as $9000, 
$15,000, and $8000, respectively. The unit transportation cost from the three 
candidate warehouses to the customers is given below: 
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CUSTOMER 
WAREHOUSE 1 2 
1 2.50 3.00 
2 3.00 3.00 
3 3.50 3.25 


Use Benders partitioning procedure of Exercise 7.38 to determine which 
warehouses to construct and the corresponding shipping pattern. 


[7.41] Consider the following linear program in » variables and having m 
constraints, where a pp j=1,...,2, and b are m—vectors: 


n 
P: Maximize XC jx; 


j=l 
: n 
subjectto > a jx; <b (7.18) 
Jel 
x20. 
Let w k= (Wr sees Wan) be a non-zero binary n—vector (i.e., having components 


equal to zero or one). Note that there are k = 1,...,2”-1=K_ such distinct 
vectors. For each k e€ {1,...,K}, define an aggregate column g Me R”™ as: 


n n 
= > a_w,., having an associated cost coefficient f, = ¥ c.w,.. 
og g I, 2 
Accordingly, formulate the following linear program: 


K 
P’: Maximize x Ay 
k=l 


K 

subjectto 8,4, <b (7.19) 
k=1 
A> 0. 


a. Show that Problem P can be equivalently solved as Problem P’. 


b. Noting that Problem P’ has an exponential number of columns (since K = 
2” —1), devise a column generation algorithm to solve it. (Hint: Given 
optimal dual variables associated with (7.19) for some restricted version of 
Problem P’ having K’ << 2”—1 columns, explicitly price the variables of 


Problem P and hence compute a most enterable column for Problem P’, if 
one exists.) 


c. How would you recover an optimal solution to Problem P from that for P’? 
(Hint: Examine the relationship between the constraint representations 
(7.18) and (7.19) to write x in terms of w and A.) 


390 Chapter 7 


d. Extend the approach of Parts a-c to the case where Problem P has bounded 
variables of the form 0 < x < 1,7 =1,...,” (possibly after scaling). (Hint: 


Note that any solution x to this Problem P can be represented as a convex 
combination of the extreme points of the unit hypercube W = {w:0<w< 


1} cR", whose 2” extreme points are defined by all possible binary 


vectors in R”.) 


[7.42] Consider the following pair of primal and dual linear programs P and D, 
where A ism x nandD is m’ x n: 


P: Minimize cx D: Maximize wb+ vd 
subject to Ax =b subject to wA+vD<c 
xe X ={x:Dx2>d,x2>0} Ww unrestricted, v 2 0. 


Let Problem BMP denote the Benders master problem defined by Equation 
(7.14). 


a. Show that the dual to Problem BMP (call this DWMP) is the Dantzig- 
Wolfe master program given by Equations (7.1) — (7.3). 


b. Now, suppose that in Problem BMP, we impose the bounds w’ < w<w 


on the w-variables, where the lower and upper bounds w~ and w” are 
estimated to (possibly) capture an optimal set of values for the w—variables. 
Explain how this might help the Benders algorithm to converge faster to an 
optimal solution. (Hint: Examine the geometry of the tangential 
approximation or outer linearization interpretation of Benders (or the 
Lagrangian relaxation) procedure discussed in Section 7.6.) 


c. Given the bounds in Part b, suppose that we find an optimal solution w* 
via Benders algorithm such that w_ <w" <w". Show that w” is (part of) 
an optimal solution to Problem D. 

d. In Part c, if the stated condition does not hold true, suggest how you might 
adjust the bounds [w ,w*] and reiterate to optimally solve Problem D 
using the Benders decomposition technique. (This is referred to as a boxstep 
method.) 

e. For the Problem BMP in Part b to which we had added the bounding 


constraints w. <w<w~ (call this BMP’), designate y* and y” as the 


dual variables associated with the constraints w<w* and -w<-w , 
respectively, and write the corresponding dual problem (call this DWMP’). 
Compare the problems DWMP and DWMP’ and comment on their relative 
structures. 


f. Using Part e, extend the analysis of the modified Benders algorithm devised 
in Parts b-d to an equivalent modified implementation of the Dantzig- 
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Wolfe decomposition method (this is referred to as a stabilized column 
generation implementation). 


[7.43] Consider the Problem BMP’ defined in Exercise 7.42, and suppose that 
we further modify this to derive the following problem: 


BMP*: Maximize z—1n*8*~x 8 (7.20) 
subject to z+w(Ax;—b)<ex; forj=1,...,f (7.21) 
w -n <wswn’ (7.22) 


(z,w) unrestricted, (xn*,x) >0, 


where nx‘ and x are variable vectors, and &* and 6& are suitable 
corresponding objective coefficient vectors that penalize any deviation of w 


outside the given bounds [w ,w’ ]. 


a. 


Discuss the relationship between Problems BMP’ and BMP‘. In particular, 
for what selection of parameter values 6* and & would BMP” reduce to 
BMP’? 

Extend your response to Parts c and d of Exercise 7.42 to the case of 
Problem BMP*. 


Associate dual variables Aj j = 1,...4, y*, and y~ with the respective 


constraints (7.21), and the pair of bounding constraints w-1n* <w* and 
-w-m <-w_ in (7.22), and write the dual to Problem BMP” (call this 


DWMP"). Translate your analysis for the modified Benders approach in 
Part b to describe a corresponding modified Dantzig-Wolfe algorithm as 


applied to Problem DWMP”. (This is a further extension of a stabilized 
column generation implementation with respect to that discussed in 
Exercise 7.42). 


NOTES AND REFERENCES 


1. 


The decomposition algorithm of this chapter is an adaptation of the 
Dantzig-Wolfe decomposition principle [1960, 1961]. The latter was 
inspired by the suggestions of Ford and Fulkerson [1958b] for solving the 
special case of multicommodity network flow problems. 

The decomposition method presented in this chapter is closely associated 
with the concepts of generalized Lagrangian multipliers, tangential approxi- 
mation of the Lagrangian dual function, and the dual cutting plane 
algorithm, as discussed in Section 7.6. For further reading on these topics 
the reader may refer to Bazaraa, Sherali and Shetty [2006], Everett 
[1963], Geoffrion [1971], Lasdon [1970], Kelley [1960], and Zangwill 
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[1969]. Exercise 7.41 describes a counterintuitive technique here to Oguz 
[2002], which creates an equivalent representation of a given general 
linear program by exponentially blowing up the number of variables and 
then applying column generation. This is shown to have some 
computational promise for the bounded-variables case. 

In addition to the Dantzig—Wolfe and similar decomposition algorithms, 
the literature covers a plethora of other decomposition methods. These 
can be classified as price—directive and resource—directive algorithms. In 
the former, a direction for modifying the Lagrangian multipliers of the 
coupling constraints is found and then a suitable step size is taken along 
this direction. See, for example, Geoffrion [1970], Lasdon [1970], Grinold 
[1972], Balas [1966b], Held, Wolfe, and Crowder [1974], Bazaraa and 
Goode [1979], Minoux [1986], Lubbecke and Desrosiers [2002], and 
Desrosiers and Lubbecke [2005]. The resource-directive algorithms 
proceed by finding a direction for modifying the shares of the common 
resources among the subproblems and then determining the step size. The 
reader may refer to Geoffrion [1970, 1974], Lasdon [1970], Abadie 
[1963], Minoux [1986], and Guignard-Spielberg [2004]. 

In Exercise 7.38 we described the partitioning scheme of Benders [19621 
that was specialized in Section 7.6. This scheme is particularly suited for 
solving mixed-integer programming problems. 

For an in-depth geometric analysis of the Dantzig—Wolfe decomposition 
algorithm, see Todd [1983]. For implementation issues, see Ho and Loute 
[1981, 1983]. Birge [1985] provides some further insights into this proce- 
dure, and Wittrock [1985] applies it to the dual of the staircase structured 
problem of Exercise 7.30. (The approach sketched in Exercise 7.30 is from 
Glassey [1973].) For generalizations of Dantzig—Wolfe/Benders algo- 
rithms, see Geoffrion [1972] and Burkard et al. [1985]. 

Marsten [1975] and Marsten et al. [1975] describe the boxstep method for 
accelerating the convergence behavior of Benders algorithm. (Related 
ideas can also be used to improve Lagrangian relaxation procedures.) 
This has inspired duality-based concepts of stabilized column generation 
for markedly improving the computational behavior of column generation 
procedures in general, and Dantzig—Wolfe decomposition in particular. 
These ideas are briefly described in Exercises 7.42 and 7.43, and are 
discussed in greater detail in the papers by duMerle et al. [1999] and 
Desaulniers et al. [2001]. 

Column generation methods are also frequently combined with branch- 
and-price techniques for solving (specially structured) mixed-integer 
programming problems. The interested reader is referred to the papers by 
Barnhart et al. [1998] and Vanderbeck [2000], as well as to a general 
review paper by Wilhelm [2001]. 


EIGHT: COMPLEXITY OF THE 
SIMPLEX ALGORITHM AND 
POLYNOMIAL-TIME ALGORITHMS 


In this chapter we discuss some fundamental computational complexity issues and 
theoretically efficient, that is to say, polynomial-time, algorithms for solving linear 
programming problems. We begin by discussing the issue of exponential versus 
polynomial-time algorithms based on a worst-case computational performance 
analysis. According to this classification scheme, the simplex algorithm is shown to 
be “bad,” because it exhibits an exponential growth in effort on certain classes of 
problems as problem size increases. Fortunately, there are some average—case 
theoretical results that help resolve the discrepancy between the worst-case 
theoretical results and the observed practical efficiency of the simplex algorithm. 
However, there do exist theoretically efficient algorithms for linear programming 
problems that exhibit a growth in solution effort that is polynomial in the size of 
the problem. The two well-publicized algorithms of Khachian and Karmarkar fall 
in this category. Although Khachian’s algorithm has failed to be of practical 
computational value, the underlying concept behind Karmarkar’s algorithm is far 
more promising. We therefore relegate the development of Khachian’s algorithm to 
the exercises (see Exercises 8.12—8.17), but treat Karmarkar’s algorithm in detail in 
this chapter. We also comment on several interior point algorithmic variants that 
have been inspired by Karmarkar’s algorithm, but that are computationally much 
more effective. Curiously, all of these algorithms are nonlinear approaches to linear 
programming problems. Although a preparation in nonlinear programming theory 
and algorithms will provide additional insights to the reader, we maintain a level of 
presentation, aided by geometric motivations, that will require a knowledge of only 
some basic linear algebra and calculus on the part of the reader. The reader is also 
encouraged to review the material in Chapter 2. 


8.1 POLYNOMIAL COMPLEXITY ISSUES 


As the field of Operations Research continued to develop following the birth of 
linear programming, a rich collection of different types of problems were intro- 
duced along with a host of competing solution algorithms. Consequently, there 
arose a need to be able to formally classify and compare problems and algorithms 
from the viewpoint of their computational tractability. To address this need, in the 
1970s, computer scientists and operations research analysts introduced the issue of 
computational complexity of problems and algorithms. The idea here is to conduct 
an evaluation of the problem or algorithm based on its performance in a worst-case 
type of situation, with an aim to ascertain how difficult a problem is to solve in the 
worst case, or what is the growth in computational effort of an algorithm as a 
function of the size of the problem in the worst case. Such an analysis provides a 
performance guarantee for an algorithm that solves a given class of problems. 
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Since our primary purpose is to relate this development with the study of 
linear programming problems, let us present some relevant issues with respect to 
this particular class of problems. Consider the linear programming optimization 
problem: 

Minimize {ex : Ax = b, x > 9}, 
where A is m X n and where m, n > 2. We assume here and throughout this 
chapter that the data is all integer, perhaps converted to this form from rational 
data. By an instance of this problem, we mean a particular member of this class 
of problems having specific parameter values or data. Hence, the specification of 
values for m, n, c, A, and b defines an instance of a linear programming 
problem. The size of an instanceof this problem is represented by the entities (m, 
n, L), where LE is the number of binary bits required to record all the data of the 
problem and is known as the input lengthof an instance of the problem. Note that 
the number of binary bits required to represent a positive integer A is | log( + 
A) |, where [-] denotes the rounded-up value and, throughout this chapter, log(-) 
denotes logarithm to the base 2. For example, in order to represent any integer 


Ae [2", 2"t!-1] for an integer r > 1, we require (r + 1) binary bits. If A is of 
either sign, we can use an extra bit to record its sign, and therefore we can 
represent A using 1+[log(1+|A}) |, bits. Consequently, the data of the foregoing 


linear program can be recorded by a binary encoding schemeusing the following 
number of bits: 


L={1+[log(1+ m) |} +{1+[log(+ ol}+z +|log(t la) } 


zt] log (1+|a,|) }. 


Consider an algorithm that finitely solves a linear programming problem. 
In order to analyze its computational complexity we are required to determine an 
upper bound on the effort required to solve any instance of this problem. This 
effort may be measured in terms of the number of elementary operations such as 
additions, multiplications, and comparisons that are required to solve the problem 
as a function of the size of the problem. Fortunately, an exact count of the total 
number of such operations is unnecessary. We are only required to determine a 
function g(m, n, L) in terms of the size of the problem, such that for some 
sufficiently large constant t > 0, the total number of elementary operations 
required by the algorithm to solve the problem is no more than q(m, n, L). In 
such a case, we say that the algorithm is of order of complexity O(g(m, n, L)). 
When the function g(m, m, L) is a polynomial in m, n, and L, the algorithm is said 
to be a polynomial-time algorithm, or of polynomial complexity or to be 
polynomially bounded For example, if an algorithm actually involves a 


aij 


> Z{I+[ tog(1+ 


maximum of, say, 6m°n + 15mn+ 12m elementary operations to solve any 
instance of some given class of problems, where the problem size is determined 


by m and n, we can simply say that it is of complexity O(m? n) (why?). Hence, in 
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lieu of deriving the foregoing exact expression for the effort, we are only 
required to ascertain that the growth in effort for this algorithm is dictated 
dominantly by the function g(m, n, L) = mn (which, in this case, is independent 
of L). 

Since we are only required to determine an upper bound on the solution 
effort, we need not compute L exactly in defining the size of an instance of a 
problem. In other words, if we take L as some /ower bound on the actual number 
of bits required to record the data and exhibit a polynomial growth in effort as a 
monotone increasing function of this quantity, then this would also imply a 
polynomial bound on the effort in terms of the actual size of the problem. Hence, 
for instance, we may take L to be the following value in our analysis of linear 
programming problems: 


1+ logm-+logn+¥|1+log(1+le,|)]+2[1+1og(1+|ay)) 
L= Z ss - (8.1) 
+¥[1+log(1+|4,|) | 


Our infatuation, or even obsession, with a polynomial-time algorithm can 
665,99 


be well appreciated by comparing the growth of n? versus 2”. Let “‘n represent 
the size of some problem, and let 4; and A, be two algorithms for this problem. 


Further suppose that Algorithm 4, is of complexity O( n) and that Algorithm 


4, is of complexity O(2”). This means that there exist constants 7; and 1 
such that the total number of elementary operations required by Algorithms <4, 


and A» are respectively bounded by the expressions qn" and 772”. 
Consequently, for a problem of size n = 50, assuming 7, = Tt = 1, although 
Algorithm 4, will perform no more than 2500 operations, the number of 


operations required by Algorithm 4, can get as high as 1.1259 x 10!5—an 
astronomical figure! There are two obvious anomalies with this system of 
evaluation. First, we would prefer low-order polynomials. A polynomial-time 
algorithm with complexity O(n? be would clearly be of no greater practical value 
than Algorithm 4,. Second, we would like the polynomial coefficient 
magnitudes to be relatively small. If 7, = 2°° and T7 = 1, then although the 
effort with Algorithm 4, blows up considerably faster with an increase in n than 
with Algorithm 4), the latter algorithm is unmanageable from a practical 
viewpoint for any value of n > 1. However, Algorithm A) is at least manageable 


for some sufficiently small values of 7. 

It is worthwhile to note the role played by the binary encoding scheme in 
defining a polynomial-time algorithm. Suppose that an algorithm was proposed 
for linear programming problems that was of order O(mnA) in complexity, where 
A is the largest coefficient in absolute value from ec, b, and A. Then this 
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algorithm is not of polynomial complexity. In fact, defining LZ, = log(A) as 
representing the size of the problem (for example, for fixed m and 7), we see that 
the algorithm is of an exponential complexity of order O(mn24 ) . Suppose that 


we had adopted to measure the size of the problem in terms of a stroke encoding 
(or unary encoding) scheme, i.e., a scheme that uses one stroke for every unit of 
data. Then to store the integer A > 0 we would require A strokes or spaces, so that 
the problem size would be determined by the triplet (m, n, A) and not by (m, n, 
log A). In this case, the algorithmic effort is indeed bounded above by a 
polynomial in (m, n, A). Such an algorithm is referred to as being pseudo— 
polynomial. We shall see later in Chapter 10 that the Hungarian algorithm for 
solving the linear assignment problem is pseudo—polynomial in complexity. 

On the other hand, an algorithm whose complexity depends only on the 
number of problem parameters, as determined for linear programs by m and n, 
for example, and is independent of the magnitudes of these parameters, is said to 
be genuinely or strongly polynomial. Whereas no strongly polynomial algorithm 
is known to exist for general linear programming problems, such algorithms 
indeed exist for the special class of network structured linear programs. 

In concluding this section, let us place the foregoing discussion in the 
context of the approach taken by computer scientists. In the domain of computer 
science, problems are usually posed as decision problems rather than optimiza- 
tion problems. For example, the linear programming optimization problem has as 
its counterpart the following linear programming decision problem: 


Given c, b, and A (of the appropriate dimensions) and given a 
rational number K, does there exist a rational vector x such that Ax 
=b, x >0, and ex < K? 


A polynomial-time algorithm for a decision problem may be (informally) 
defined as one that can “solve” any instance of the problem using a number of 
elementary operations bounded above by some polynomial in the size of the 
problem. By “solve,” we mean halt with an answer yes, given a yes—instance of 
the decision problem. Hence, if we have a polynomial-time algorithm, and we 
are given a no-instance of a problem, it is sufficient to exceed the established 
polynomial bound without triggering a suitable termination criterion to conclude 
that the answer is no. Therefore, the polynomial bound on the effort required to 
solve the problem holds true for all instances. Decision problems for which some 
polynomial-time algorithm is known to exist are said to belong to the 
distinguished Class P of problems. 

It is fruitful to see the relationship between polynomial-time algorithms 
for optimization problems and those for decision problems. A polynomial-time 
algorithm for an optimization problem can clearly solve a decision problem in 
polynomial time (how?). Conversely, suppose that we have a polynomial-time 
algorithm for the linear programming decision problem. Consider an instance of 
the optimization problem that is feasible and bounded. Then there exists a basic 


feasible solution of objective value < 2+ in magnitude (see Exercise 8.11). 


Hence, the optimal value lies in the interval Pg oy ]. Assuming that the algo- 


Complexity of the Simplex Algorithm and Polynomial-Time Algorithms 397 


rithm for solving the decision problem also actually produces a solution for a 
yes—instance, we can use this algorithm to solve the optimization problem by 
repeatedly performing a bisection search on this interval. This can be done as 
follows: 

First, using K = 0, which is the midpoint of the current interval of 


uncertainty [ooh oF), it can be found in polynomial time whether or not the 


optimal value lies in the interval pea , 0] or [0,2” ]. Using the appropriate 
interval that contains the optimal value, this interval of uncertainty can again be 
bisected; this process can be continued until, say, the optimal value is known to 


lie in the interval [v,, v2] where vy —vy, < 2°*/. Since the initial interval of uncer- 


tainty is of length 2'*| and this is halved at each bisection, we can obtain this 


reduced interval of uncertainty after using 3L + 2 applications of the decision 
problem algorithm (why?). This means that we will have used a polynomial-time 
algorithm a polynomial number of times, and hence, we will have accomplished 
this in polynomial time. If a corresponding solution in this interval is not yet 
available, it can be obtained by applying the decision problem with the task of 
finding a solution x having ex < v). As will be shown later in Section 8.5, given 


a solution with this accuracy in objective value, we can apply a polynomial time 
“rounding scheme” in order to determine an optimal solution to the optimization 
problem. Hence, the optimization problem can be solved in polynomial time by 
using a polynomial-time algorithm for the decision problem. 

We now proceed to demonstrate that the simplex algorithm is of 
exponential complexity, and then present in detail a particular polynomial-time 
algorithm for solving linear programming problems. 


8.2 COMPUTATIONAL COMPLEXITY OF THE 
SIMPLEX ALGORITHM 


When Dantzig first introduced the simplex algorithm, the intuition—based 
reaction of the research community was that this algorithm would not prove to be 
very efficient. By its nature, the simplex algorithm crawls along the edges of a 
polyhedron and makes no attempt to skip through its interior or over higher 
dimensional faces. In fact, it occasionally stalls during a sequence of degenerate 
pivots at some extreme point. However, researchers were pleasantly surprised 
when, in practice, this method performed exceedingly well. For most practical 
problems, this method has been empirically observed to take roughly 37/2 
iterations, and seldom more than 3m iterations, where the coefficient matrix A is 
m x n. As pointed out in Chapter 5, a regression equation of the type 


Km? nd°?, for example, where d is the density of the matrix A, may typically 
predict the performance of the simplex algorithm on a given problem quite well. 
However, the fact remains that the algorithm is indeed entrapped in the 


potentially combinatorial aspect of having to examine up to (7) vertices that the 


algorithm could possibly visit. Since 
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whenever n > m, the potential of an exponential order of effort for some 
problems is quite plausible. 

The fear of exponential effort with the simplex method was confirmed in 
1971 when Victor Klee and George Minty produced a class of problems defined 
by some m = n equality constraints in 2m nonnegative variables, for which the 


simplex algorithm requires 2” —1 iterations, traversing all the vertices of the 
problem. This class of problems results by defining the feasible region as a 


suitable distortion of the n—dimensional hypercube in R”, which has 2” vertices. 


Mathematically, the problem in R” may be stated as follows, where ¢ is some 
rational number in the interval (0, 1/2): 


Maximize x, 
subjectto O<x, <1 
Bxjj Sep lex for j =2,...,n 
x;2 0, j=l,..in. 
After suitably transforming this problem, we exhibit that, using Dantzig’s rule of 
entering the nonbasic variable having the largest reduced cost and starting at the 


origin, the simplex algorithm traverses 2” —1 edges of this polytope and visits all 


the 2” vertices. 
In order to restate this problem in a desired form, consider the linear 
transformation: 


Y= MV; =(x,-ex,,)/elt for j = 2,...,7. (8.2) 
Using this transformation and putting @= 1/é, we obtain the following equivalent 


problem. (The algebraic details are left to the reader in Exercise 8.5.) 


n 
Maximize >» y; 

j=l 
subject toy, <1 


pe . (8.3) 
yy+2D Vy sof for j =2,...,.7 

k=l 
Vo Vy 29. 


For convenience, define s),...,5, as the slack variables associated with the n 


n 
structural constraints in Problem (8.3). Observe that the basic solution having 
basic variables (s;,...,5,-1,¥,) is feasible, yielding y = (0,...,0,0"1) with objec- 


tive function value 671. Noting the final structural constraint and the objective 
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yy 


(a) Case for n = 2 (b) Case for n =3 
Figure 8.1. Illustration of the Klee—Minty type polytopes for n=2 and n=3. 


function in Problem (8.3), this clearly corresponds to an optimal solution to this 
problem (why?). In Exercise 8.4 we ask the reader to verify this by computing 
the (z pe; )-values for the nonbasic variables. 


Let us now consider the case n = 2. The problem in this case is of the form 
to maximize y, + y2, subject to y, <1, yo +2y, <9, and y,, yy 20. Figure 8.la 
shows the feasible region using @= 3. Note that for the origin, the basic variables 
are (51,52). Hence, the (z;—c;)—values for y, and y2 are both —1. Selecting 
y, as the entering variable takes us to the adjacent extreme point solution with 
basis (31,52). Continuing, we now follow the simplex path {A, B, C, D} shown 
in Figure 8.la. In doing so, we traverse the 27 basic feasible solutions having 
respective basic variables (s},5>), (31552), (4,2), and finally (s,,y¥2). The 


reader may verify that z;-c,; = —1 for the entering variable at each iteration. 


Note that if we had broken the ¢ie at the first iteration by selecting y, as the 


entering variable, we would have solved this problem in one iteration. 
Next, consider the case n = 3. The problem in this case is of the form to 


maximize y) + y+ y3, subject to y, <1, yo +2y, SO, ¥34+2y, +2y2 S , and 
¥|>¥2+¥3 20. Figure 8.1b shows the feasible region using @ = 3 as previously. 
Now, let us make the following observation. First, notice that with y3; = 0, the 


400 Chapter 8 


extreme points in the (),, 2) space obtained for the case n = 2 are still extreme 
points in the present case; s3 is the third basic variable. The corresponding sets 
of basic variables are (5),57,53), (91,52553)s (52.53), and (s), y2,83). Second, 


observe that in moving from each one of these bases to the next, the (z Bee Fs as 
value for the entering variable is —1 as for the case n = 2. The (z; —c;)—value 
for the new nonbasic variable y, also remains at —1 throughout, since the last 


component of the simplex multiplier vector ¢ 3B! is zero for all these bases B. 


Hence, the simplex path {A, B, C, D} for the case n = 2 remains a simplex path 
for the case n = 3 under Dantzig’s entering rule. Moreover, y3 remains enterable 


throughout. Third, at the vertex D, we must enter y3, and because s3 and y3 can- 
not both be basic, s3 must leave the basis. For the remaining iterations, y3 
remains basic. Using the last equation to solve for y3 in terms of y,, y>, and 53 
and eliminating y; from the problem gives the same problem as for the case n = 
2, except that the objective coefficients of y, and y, are now —1. In other 


words, it is as if the n = 2 problem has been changed from a maximization to a 
minimization problem at the end of its solution! Consequently, maintaining 
Dantzig’s entering criterion, the simplex path simply reverses through the bases 
for the case n = 2, with y3 being the additional basic variable throughout. We 


now visit the vertices E, F, G, and H in Figure 8.15, with respective sets of basic 
variables (51, 25.3), (¥1,¥25¥3)> (1155253), and (s,82, 3). Therefore, in solving 
this problem, we have traversed Ps edges and visited (all the) 23 extreme 


points in the process. 
This is now readily generalized using an inductive argument. For example, 


for the case n = 4, we first go through the set of 2? bases having basic variables 
(51 552553,54)5 (¥1552593594), (1s ¥2+93554),  (515.2553,54), (515925 ¥3>54)s 
(115¥25V3284)s (1458253554), and (s,,52,¥3,54). This sequence is obtained by 
using the n = 3 case with s4 as the additional basic variable. Then, y4 enters and 
54 leaves the basis, giving the set of basic variables (s,,52, ¥3,¥4). The 
sequence of pivots reverses with the maximization problem for the n = 3 case 
effectively turning into a minimization problem when y,4 is held basic. We next 
generate the 23 bases with basic variables (51582,¥3,¥4)s Os $25¥35¥4)s 
(¥15¥25¥35¥4)s (S1s¥25V3sV4)s (Sp5Y29832¥4)s 1s ¥2953:¥4)s is 529534), 
and (s),82,53,4). This results in 2x23 =2* bases over 24-1 pivots. The 


construction of the formal inductive proof is now straightforward and is left to 
the reader in Exercise 8.6. It is interesting to observe that for the general case 7, 


all throughout the first 2”! basic feasible solutions, the variable y, remains 
enterable. Recall the definition of a stage from Chapter 4! Also, notice that at the 
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end of 2”! pivots, the basic variables are (5),52,.--,5)-2 Vn—1>¥n)- If 5,1 is 
entered at this pivot, then the optimum results. (This corresponds to going from E 


to H in Figure 8.15.) However, both y, and s,_; have z;—c; = ~1 here, and the 


choice of entering y, takes us through the reverse sequence of pivots. 
Notice the exorbitant amount of effort as problem size increases. Assume 


that the problem having n constraints in 2n nonnegative variables requires 0.25n" 
elementary operations per pivot, and that each operation takes one nanosecond 


(lo~° sec) to perform. Then, the estimated computational effort for = 50 is 


22.3 years; for n = 60 it is over 328 centuries! One may argue that the simplex 
algorithm did have an opportunity to solve the foregoing problem in one 
iteration. In fact, by scaling the columns, for example, using the transformation 


¥, = pas y;, we can eliminate the ties in the entering criterion at the first iteration 


and thereby solve the problem in one iteration unarguably. Observe also that the 
rule of entering the variable that gives the maximum improvement in the 
objective function value also solves this problem unambiguously in one iteration. 
Moreover, this criterion is invariant with respect to scaling and problem 
representation, since it is intimately connected with the structure of the polytope 
itself. In 1973, however, Robert Jeroslow exhibited the existence of problem 
classes that yield an exponential effort with the maximum-improvement entering 
criterion as well. In fact, for some fixed integer r > 1, if one were to perform all r 
next pivots, and select the r-pivot simplex path that gives the best improvement 
in objective value, then the algorithm would still exhibit an exponential trend on 
some classes of problems. The root of the problem is therefore in the myopic or 
local viewpoint of the simplex algorithm whereby decisions are based on the 
local combinatorial structure of the polytope, and where the motion is restricted 
to an edge path. 


8.3 KHACHIAN’S ELLIPSOID ALGORITHM 


Confronted with the exponential worst-case behavior of the simplex algorithm, 
the question of whether there existed a polynomial-time algorithm for solving 
linear programming problems remained open until 1979, when it was answered 
affirmatively by L. G. Khachian. Khachian proposed a polynomial-time 
algorithm for determining a solution (if one exists) to the open set of linear 
inequalities S= {x : Gx < g}, where G is r x g with r and g > 2, and where G and 
g have all integer components. This algorithm either finds a solution in S, if one 
exists, or else concludes that S is empty. Its basic mode of operation is as 
follows: 


First, the method defines a ball or sphere in R?, centered at the origin and 
having a radius that is large enough to encompass a sufficiently large volume of 
S, assuming that S is nonempty. If the center of this ball is in S, then the 
algorithm terminates. Otherwise, the method proceeds to construct a suitable 
sequence of ellipsoids that are monotonically shrinking in volume, but all of 
which contain the region of S that was captured by the initial ball. Whenever the 
center of any ellipsoid is found to lie in S, the algorithm terminates. The principal 
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result states that if S is nonempty, then the center of some ellipsoid will lie in S 
within a number of iterations that is bounded above by a polynomial in the size 
of the problem. Moreover, if this bound on the number of iterations is exceeded, 
then the algorithm can terminate with the assertion that S is indeed empty. 
Exercises 8.12-8.17 deal with the details of this algorithm. How such an 
algorithm helps solve the linear programming (optimization) problem in 
polynomial time is addressed in Exercises 8.20—8.23. In particular, it is shown 
that the linear programming problem to minimize {cx : Ax > b, x > 0}, where A 
is m x n, can be solved by this algorithm with an effort of complexity O[(m + 
n)° L]}, where L is given by Equation (8.1). 

Unfortunately, computational experience with Khachian’s algorithm and 
its variants has been very disappointing. This has contributed to the realization 
that one ought to consider as “efficient” only low-order polynomial algorithms, 
perhaps those that are also strongly or genuinely polynomial. The practical per- 
formance of Khachian’s algorithm is strongly connected with its theoretical 
worst-case bound. The example in Exercise 8.24 exhibits the strong dependency 
of algorithmic effort on the magnitudes of the input data. Hence, the initial 
excitement that this algorithm generated, both from the theoretical viewpoint and 
from the simplicity of its steps (which are programmable even on a hand 
calculator), fizzled out as it was realized that even problems having about 100 
variables can require an enormous amount of effort. Today, Khachian’s 
algorithm serves mainly as a theoretical tool to analyze complexity issues related 
with various row generation procedures for combinatorial optimization problems. 
We encourage the reader who is interested in the theoretical aspects of this 
algorithm, either from a linear or a discrete/nonlinear programming viewpoint, to 
study Exercises 8.12-8.17 and 8.20—8.23, as well as the relevant papers cited in 
the Notes and References section. 


8.4 KARMARKAR’S PROJECTIVE ALGORITHM 


The simplex method, having undergone considerable refinement and sophistica- 
tion in implementation, had no serious competition until 1984 when N. Karmarkar at 
AT&T Bell Laboratories proposed a new polynomial-time algorithm for linear 
programming problems. This algorithm addresses linear programming problems 
of the following form: 


Minimize cx 


subject to ae zs ' (8.4) 
x > 0, 


where A is m x n of rank m, n > 2, ¢ and A are all integers, 1 is a row vector of n 
ones, and where the following assumptions (Al) and (A2) hold: 


t 
(A1) — The point xg = [...4} is feasible in Problem (8.4). 
non 
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(0, 0, 1) 


(0, 1, 0) 


Figure 8.2. Projective transformation of the feasible region. 


(A2) — The optimal objective value of Problem (8.4) is zero. 


At first glance, the form of the linear program (8.4) and the accompany- 
ing assumptions (Al) and (A2) may appear to be unduly restrictive. However, as 
shown later, any general linear programming problem can be (polynomially) cast 
in this form through the use of artificial variables, an artificial bounding 
constraint, and through variable redefinitions, if necessary. Moreover, we will 
show that the analysis under these conditions can be readily extended to 
polynomially solve the general linear programming problem. Let us therefore 
proceed with the derivation of a polynomial-time algorithm to solve Problem 
(8.4) under assumptions (Al) and (A2). Note that under these assumptions, 
Problem (8.4) is feasible and bounded, and hence, has an optimum. 


Starting with xg =(1/7,...,1/n)' and k = 0, the algorithm performs the fol- 
lowing iterative steps, given any feasible solution x; > 0. Define the diagonal matrix 


D, = diag{xg1,....Xg,}, where x; ={x,),...,X,,}', and consider the following vari- 
able transformation: 


D;!x 


that is, 


Xj / Xp; 
n 
DX (x; / xy) 
j=l 
Observe that the feasible region in Problem (8.4) is described by the intersection 
of the (7 ~ m)-dimensional linear subspace defined by the homogeneous set of 


equalities Ax = 0 with the (7 — 1)-dimensional simplex S, = {x : 1x = 1, x > 0}. 


y= fori =1,...,n. (8.5a) 


The intersection is some (” — m — 1)-dimensional region. Figure 8.2 illustrates a 
situation in m = 3 dimensions with a single (m = 1) homogeneous equality 
constraint. Under the transformation (8.5a), any point in S, is transformed into a 
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point in the (7 — 1)-dimensional simplex S,, = {y: ly = 1, y 2 0}. In particular, 


the current point x; is transformed into the point yg = (1/v,...,1/ n)', the center 
of the simplex Sy (see Figure 8.2). Note that if we were to have used the affine 


transformation y'=D;!x, which is simply a scaling operation, then a point 
x eS, would, for example, transform into a point y’ as shown in Figure 8.3. If 
this point y’ is projected onto S,, along the ray defined by the origin and y’, 
then we would obtain the point y given by the transformation (8.5a), as shown in 
Figure 8.3. This amounts to dividing y’ by the sum of its components, as in 


transformation (8.5a), so that the components of y sum to one. For this reason, 
the transformation (8.5a) is known as a projective transformation. Performing 
this operation on the feasible region in the x-space results in the feasible region 
in the y-space as illustrated in Figure 8.2. 


To algebraically describe this feasible region in the y-space, consider the inverse 
projective transformation obtained by solving for x in S, from Equation (8.5a). 
This yields x = (D,y)(1D;'x). Since 1x = | in S,, we have (1D, y) (1D;!x) = 1. 
Hence, (1D;'x) =1/(D,y). Substituting this into the previous expression for x 
yields 

x= Diy ; 

1D;y 

Therefore, the transformation (8.5a, b) maps points from the simplex S, onto the 


(8.5b) 


simplex S,,, and vice versa. 


Under the transformation (8.5a, b), the linear program (8.4) equivalently 
becomes the following problem in the y—space over a subset of the simplex S,;: 


D 
Minimize aoe :AD,;y =0,ly =l,y 20>. (8.6) 
IDyy 


Note that although the constraints remain linear because Ax = 0 is homogeneous, 
the objective function has been transformed into a quotient of linear functions. 
This type of problem is known as a linear fractional programming problem. 
However, by Assumption (A2), the optimal objective value in Equation (8.6) is 
zero. Hence, we can equivalently minimize the numerator in the problem, since 
the denominator is positive and bounded away from zero for all y <S,,. This 


leads to the following problem: 


Minimize{cy : Py = Po, y 2 0}, (8.7) 


Complexity of the Simplex Algorithm and Polynomial-Time Algorithms 405 


Figure 8.3. Projective transformation. 


Byg.rOS, 


Byo,aryasy 


Ynew 


Figure 8.4. Main step of Karmarkar’s algorithm. 


AD 0 
t=cD,, P=| il and r= [t 


Rather than solve Problem (8.7), which is equivalent to solving the origi- 
nal problem, we will optimize a particular simpler restriction of this problem. 
This will yield a new feasible solution. By iteratively repeating this step, we shall 
be able to polynomially obtain an optimal solution to the original problem. 


where 
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B(yo,@r) aty :Py= Po} 


Yo 


Byg,aryas,, 


Py = Py 


Figure 8.5. Illustration of Karmarkar’s algorithm. 


To define this restriction, consider the n-dimensional sphere or ball B(y,7) 


with center at yg =(1/n,...,1/n)' and with the appropriate radius r such that the 


intersection of this ball with the simplex constraint {y : ly = 1} is an (n — 1) 
dimensional ball having the same center and radius, which is inscribed within 
Sy (see Figure 8.4). Note that this radius r is the distance from the center 


(1/n,...,1/n)' of the simplex to the center of one of its facets. But the facets of 
S,, are simply one lower dimensional simplices. Hence, considering the facet at 


which the component y, is zero, for example, we obtain r as the distance from 


(1/n,...,1/n)! to (0, 1/(n- 1),...1/(n— 1))'. This gives r= 1/.{n(n-). 

Consider a restriction of Equation (8.7) in which we seek to minimize ty 
subject to Py = Py, y > 0, and y € B(yp,ar), where 0 < a@< 1 is some constant. 
The specific choice of @ is addressed later. Note that we have shrunk the 
inscribed ball by the factor @, and so its radius is ar. Moreover, by remaining 
feasible to this restriction, the iterates remain strictly positive. Since y > 0 is 
implied by the intersection of the ball and the simplex constraint {y : ly = 1}, this 
restriction is equivalent to the problem: 

Minimize {éy : Py = Py,(y-Yyo)' (Y-Yo) $@-r}, (8.8) 


where {y : (y -Yo)' (y-Yo) < ar} describes the ball B(yo,ar). Figure 8.4 

depicts the feasible region over the two-dimensional simplex (for 7 = 3). 
Observe that the system Py = Po defines an (” — m — 1)dimensional 

affine subspace that passes through the center of the ball B(yg,@r). Hence, the 


feasible region in Problem (8.8) is an (7 — m — 1}-dimensional ball centered at 
Yo. Figure 8.5 illustrates a situation for n = 4, and m = 1 over ann - 1 =3 


dimensional simplex. It should be evident that the optimal solution to Problem 
(8.8) is obtained by simply projecting the negative gradient of the objective 


function —c’, centered at Yo. onto the null space, or the constraint surface, of 
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Py =Py, and then moving from yg along this projected direction to the 
boundary of the ball B(yo,ar). (See Figures 8.4 and 8.5.) Observe that the 


simplicity of this solution process is a consequence of the fact that we are 
optimizing a linear function over a ball, and that we are currently located at the 


center of this ball. Denoting the projection of the gradient vector t’ as (the 
column vector) ¢,, and the optimum to Problem (8.8) as Yneyw, we get 


p 
Ynew =Yo—@r7—- (8.9a) 
lel 
{Note that if ¢ p = 9, then any feasible solution is optimal (why?), and we may 
terminate with x, as an optimal solution to Problem (8.4).] 


In order to calculate ¢,, observe that since ¢ : lies on the constraint 


pP 
surface of Py = Po, the vector (¢’ —¢ p) is contained in the space spanned by the 
gradients to Py = Pp. Hence, there exists a vector W such that Pw =c' -c in 
Multiplying both sides by P, we get PP’ w = Pe’, because Pe p = 9 by definition. 
Noting that A is of full row rank and that x, > 0, the square matrix PP’ is 
invertible (see Exercise 8.25). This yields W =(PP’ yl pe’. Hence, we have 


c, = ¢' — Pw given as 


¢, = [1-P' PP’)! ple. (8.9b) 


In Exercise (8.26), we ask the reader to use an alternative approach to 
show that the optimal solution to Problem (8.8) is given by Equations (8.9a, b). 
We remark here that the solution W_ is the solution to the /east-squares problem 


of minimizing le - iw] over we RO™*) (see Exercise 8.27). Hence, in lieu of 
explicitly computing ¢ p using Equation (8.9b), we may obtain ¢,, by deriving a 
solution w to this problem by using any nonlinear search method, and then 
setting ¢, = ¢’ —P’w. In either case, the formula (8.9b) gives ¢ p 

Given Yyew in Equation (8.9a), the corresponding revised vector x;,,, in 
the x—space is obtained via the inverse transformation (8.5b) as 


D 
Xp = new, (8.10) 
1D. Ynew 


Observe that yney > 0 since it lies in the interior of the (x — 1)-dimensional 
sphere inscribed in S,. Hence, x,,, > 0 in Equation (8.10). This completes one 


iteration. The process may now be repeated after incrementing k by one. As 
shown in Section 8.5, the objective value in Problem (8.4) of the sequence of 
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iterates thus generated approaches the optimal objective value of zero in the limit 
as k-—>oo, although not necessarily monotonically. Hence, for practical 
purposes, this process may be terminated when the objective value gets suffi- 
ciently close to zero. 

For theoretical purposes, consider the following lower bound L on the 
input length for Problem (8.4): 


L= |! + log(I +c; max|) + log [det max|) | 


where is the largest numerical value of any cost coefficient c j> and 


le; max 
|det nax| is the largest numerical value of the determinant of any basis for Prob- 


lem (8.4). It follows from a property of the determinants stated in Exercise 8.9 
that L is indeed a lower bound on the input length for Problem (8.4), since 


log (|det nax|) S log(1+m)+¥;d ; log(1 +|aj)). (Note that an alternative, although 


larger, lower bounding value of L is given by replacing log (|det max |) with the 
latter term, as illustrated in Example 8.2.) In the next section, we shall 
demonstrate that by using a value of a@ = (n — 1)/3n, the algorithm produces a 
solution of objective value smaller than 2" within 10nL iterations, with an 
overall effort of polynomial complexity O(n? 51). Once this occurs, an exact 


extreme point optimal solution to Problem (8.4) may be obtained via the 
following polynomial-time procedure. 

Starting with the final iterate x, obtained with objective value ex, < ot: 
this procedure finds an extreme point solution with at least as good an objective 
value using the following method known as a purification scheme. If n linearly 
independent constraints are binding at x;,, then it is already a basic feasible 
solution, Otherwise, there exists a direction d ¥ 0 lying in the null space of the 
binding constraints, that is, satisfying the homogeneous equality system 
corresponding to the binding constraints. The method now moves the current 
iterate along the direction d if ed < 0, and along the direction —d otherwise, until 
some constraint blocks any further motion by feasibility considerations. This 
motion must be blocked because the feasible region is bounded. Note that at the 


new solution, the objective value is no more than ex; <2-" and at least one 


additional linearly independent constraint is binding. Proceeding in this fashion, 
a basic feasible solution x to Problem (8.4) can be obtained having an objective 


value strictly less than a. 
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x] 
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Figure 8.6. IHustration for Example 8.1. 


Note that this process involves at most ” — (m+ 1) such steps, since it 
begins with the (m + 1) linearly independent equality constraints in Problem (8.4) 
binding and then adds at least one additional linearly independent binding 
constraint to this set at each step. Moreover, note that each step is of polynomial 
complexity (why?). Hence, x is determined from the final iterate x, in polyno- 


mial time. (See Exercise 8.31 for efficiently implementing this process with a 
polynomial bound of O(m?n).) 


We now show that this resulting extreme point solution x is indeed 
optimal for Problem (8.4). For this reason, the process is known as an optimal 
rounding routine. Since the data is all integer, any basic feasible solution for 


Problem (8.4) with basis B has objective value “cpB |b” given by N/D, where N 
and D are integers, and where D= |det B (why?). But D< 2", where L is 


defined as previously. Hence, by Assumption (A2), any nonoptimal extreme 
point solution for Problem (8.4) must have a positive objective value N/D > 1/D 


>2-/. But because 0 < ex < are we must therefore have cx = 0, i.e., K is an 
optimal solution to Problem (8.4). 


Summary of Karmarkar’s Algorithm 
INITIALIZATION 
Compute r=1/Jn(n—-l), L= [1+loa(1+[¢; max|) +108 (|detmax|) | and select 


a =(n—1)/3n. Let x9 =(1/n,...,1/n)! and put k= 0. 
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MAIN STEP 


If ex; < 2, use the optimal rounding routine to determine an optimal solution, 


and stop. (Practically, since ot may be very small, one may terminate when 
cx; is less than some other desired tolerance.) Otherwise, define 


t 
. 1 1 

D, = diag{x;),....X;)}, Yo -(4...4) ; 
n n 
P =A] and © = cD, 


and compute 


Yoow =Yo-are |, where Cp = [1-P (PP y'p |e. 
lep| 


Hence, obtain x;4; =(DzYpew )/(1DzY new ). Increment & by one and repeat the 
Main Step. 


OPTIMAL ROUNDING ROUTINE 
Starting with x;, determine an extreme point solution x for Problem (8.4) with 


eX $cx;, < a*. using the earlier purification scheme. Terminate with X as an 
optimal solution to Problem (8.4). 


Example 8.1 


Consider the linear programming problem: 


Minimize x 


subjectto x, + x, -— 2x3 = 0 
Hy +t xX t+ B= 1 
x}; x2, x3 > 0. 


Here, n = 3 and m = 1. The feasible region of dimension n — m — 1 = 1 is 
illustrated in Figure 8.6. Clearly, the point xg = (1/3,1/3,1/3)’ is feasible and the 
solution (2/3,0, 1/3)" is optimal with objective value zero. Hence, Assumptions 
(Al) and (A2) are satisfied. We also have r =1/ V6 and a =2/9. 
Iteration 1 
Starting with xq and & = 0, we define Dg = diag(1/3, 1/3, 1/3). Noting that 
1D;'x = )3x; = 3, the projective transformation (8.5a) gives y = x. Hence, the 
problem in the y-space coincides with that of Figure 8.6. The current point in the 
y-space, as always, is yg = (1/3,1/3, 1/3)’. 

We now compute ¢ =cD, =(0,1/3,0). For use in Equation (8.9b), we 
compute the following using A = [1, 1, —2]: 
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pe) APE) =| us oa, pr’ =| 7/5 a (pP’)"! (#2 a 


1/2 72 0 
pp! p=] /2 iat Ppp’)! p=|1/2 1/2 01. 
From Equation (8.9b), we obtain 


crete YP ofa) is} fen = Je +16 +0 = v2/6. 
0 0 Oj; 0 


Observe that the direction —e, takes us from (1/ 3,1/3,1/3)' toward the optimal 
solution (2/3,0,1/ . in this case. Using Equation (8.9a), this gives us 
-1/6] [0.397484 
Ynew = i - (2/9)(1/-V6)(6/V2)} 1/6 | =| 0.269183 |. 
0 0.333333 


Transforming into - x~space via Equation (8.10), we obtain x; =Yyew for this 
first iteration. The present objective value is ex, = 0.269183. 


Iteration 2 


We now have k= 1. We define 
D, = diag {0.397484, 0.269183, 0.333333}, ¢=cD, = (0, 0.269183, 0) 
and 


lr cata 0.269183 ee 
Sale, A chee 1 1 1 


From Equation (8.9b), we compute 
—0.1324029 
=[1-Ptcpp’y'p [e' =| 01505548} and —_[e, |= 0.2013121. 
—0.0181517 
Substituting into Equation (8.9a) gives, using ar/ le p | = 0.4506524, that 
1/3 —0.1324029 0.3930009 
Ynew =| 1/3 |-0.4506524| 0.1505548 | =| 0.2654855 |. 
1/3 —0.0181517 0.3415134 
From Equation (8.10), we obtain 


0.1562112 
DiYnew =| 9.0714641 |, ID1Y pew = 0.341513, 
0.1138378 


0.457409 
D 
{anew saonse| 


IiYnew | 0.333333 
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This completes the second iteration. The current objective function value is ex, = 
0.209258. 

It should be evident that the iterates x, being generated are of the type 
(1/3+9, 1/3—6, 1/3)' where @ is gradually increasing. In fact, by carrying out 
these computations, it may be verified that @ approaches 1/3 in the limit as 


k +o. Note that L =[1 + log(2) + log(3) | = 4, and so, 2+ = 0.0625 for our 
problem. Hence, theoretically, we may halt the computations when ex; < 


0.0625. This happens for k = 6, for which we obtain x¢ = (0.606509, 0.060158, 


0.333333)’, with ex, = 0.060158. 


Finally, let us optimally round this solution. Note that only the two 
(linearly independent) equality constraints in the problem are binding at X¢. 


Hence, we need to first determine a solution d=(d,,d),d;)#0 to the 
homogeneous system d, +d, —2d; =0 and d, +d, +d; =0. Solving for d, and 
d, in terms of dy, we get d| =—d> and d;, =0. Arbitrarily taking d) = 1 gives 
the direction d=(-1, 1, 0)’. Since ed = 1 > 0, we move along the direction —d 
= (1, —1, 0)’. The constraint X2 = 0 blocks the motion along this direction after 
a step length of 0.060158 and yields the required third linearly independent con- 
straint. The new solution X = (0.666667, 0, 0.333333)’ is therefore an extreme 
point solution, which is moreover optimal. 

Example 8.2 


As another illustration of the computations and the progress of the iterates in 
Karmarkar’s algorithm, consider the linear program of the form (8.4): 


Minimize -—x, — 2x + 4x; 
subjectto x — Xx) + 2x3 - 2x5 = 0 
x + 2x + + x4 — 4x5 = 0 
XH + Xp + Hy + Xe +t XS = 1 
x 2 0. 


Note that the solution x9 = (1/5,1/5,1/5,1/5,1/5)’ is feasible here. Observe from 
the second constraint that the objective value is nonnegative for all feasible 
solutions, while it is zero for the feasible solution x* = (0,2/5,2/5,0,1/5)’. 
Hence, x” is optimal with objective value zero, and so, the required assumptions 
(Al) and (A2) hold true. 

We now have n = 5, so r= 1/20 and a = 4/15. Following the earlier 


suggestion of using log(1+ m)+¥;¥ ;log(+|a ) in lieu of having to compute 


i| 
log (|det max |), we have L =[ 1 + log(5) + log(3) + {4 log(2) + 3 log(3) + log(5)} 1= 
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16, so that 2+ =0,00001526. We start with the known feasible, positive solution 
Xo with objective value 0.2. 


Iteration 1 
The following quantities are computed: 
Do = diag {0.2,0.2,0.2,0.2,0.2}, yo =(0.2,0.2,0.2,0.2,0.2)', 
¢ = (-0.2,—-0.4,0,0,0.8), 


0.2 -0.2 04 0.0 -04 
P=|02 04 0.0 0.2 -0.8), 
10 10 10 10 1.0 


0.6947 -0.1474 -0.3754 -0.2175 0.0456 
-0.1474 0.2737 0.2211 _-0.3579 0.0105 

[1-Prppty'P |= ~0.3754 0.2211 0.2854 -0.1181 -0.0129 
-0.2175 -0.3579 -0.1181 0.7415 -0.0480 

0.0456 0.0105 -0.0129 -0.0480 0.0047 


¢, = (0.0435, -0.07 16, 0.0236, 0.1483, 0.0096)" , le p|=0.1722 


are, ; 
Ynew = Yo Fel = (0.2151, 0.2248, 0.2082, 0.1487, 0.2033)’. 
P 


Hence, 


x= DoY new 
1 new 


IDoY new 
for this first iteration, and ex, = 0.1487. This is now repeated with x, leading to 
the sequence 


X> = (0.2268, 0.2446, 0.2149, 0.1078, 0.2059)’ with ex, = 0.1078, 


X3 = (0.2355, 0.2596, 0.2200, 0.0769, 0.2079)! with ex; = 0.0769, 
until we obtain 


X76 = (0.2569681, 0.2972037, 0.2329655, 0.0000150, 0.2128477)' 
with exy¢ = 0.000015. 


Note that ex < 2-£ = 0.00001526 at termination. 

To perform an optimal rounding, note that corresponding to X = X¢, the 
three (linearly independent) equality constraints in the problem are binding. 
Hence, we need to determine a nonzero solution to the homogeneous system: 
d, —d3 + 2d; —2d; =0, d+ 2d, +d4—4ds =0, and d, +d,+d,;+d,+d5 =0. 
Solving for d,, 4), and d3 in terms of d, and ds, we get dj =d4+20ds, 
dy =—d4—8ds, and d, =—d4 —13ds. Arbitrarily setting dy =1 and d; =0 gives 
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the direction d=(I, —1, -1, 1, oy’. Since cd = 1 > 0, we move along the 
direction —d. This motion is blocked by the constraint x4 20 at a step length of 
0.000015, which leads to the new solution x = (0.2569531, 0.2972187, 
0.2329805, 0, 0.2128477)'. Observe that cx = 0, and so we are incidentally 
already at an optimal solution, but one that is nonextremal. Continuing, we now 
have four linearly independent constraints binding. Adding the constraint d, = 0 
to this homogeneous system, we obtain d; = 20d;, d, = —8ds, and d3 = —-13ds, 
in terms of ds. Arbitrarily setting d; = 1 gives the direction d = (20, —8, —13, 0, 


1)’. Noting that ed = 0, we can move along this direction d. The constraint x3 > 
0 blocks the motion along this direction at a step length of 0.017921576, which 
yields the revised solution X¥ = (0.6153846, 0.1538461, 0, 0, 0.2307692)’. 


Because there are five linearly independent constraints binding at this new 
solution X, it is an optimal extreme point solution. Note that if we had moved 
along the direction —d at the previous step, we would have reached the solution 


(0, 0.4, 0.4, 0, 0.2), which happens to be the other of the two alternative optimal 
extreme point solutions for this problem. 


Converting a General Linear Program into Karmarkar’s Form 


Consider a general linear programming problem: 
Minimize {ex: Ax =b,x > 0}, (8.11) 


where A is m x n of rank m and the data is all integer. We shall now describe 
how to convert this problem into the form of Problem (8.4) required by 
Karmarkar, while satisfying assumptions (Al) and (A2). (See Exercises 8.42 and 
8.45 for alternative techniques.) 

Toward this end, let us begin by first regularizing Problem (8.11), that is, 


adding a bounding constraint Lia% ; <Q. Here, Q may be taken as some 
known (hopefully small) integer bound on the sum of the variables, derived from 
feasibility and/or optimality considerations. Noting Exercise 8.10, we can take 
O= 2" in the worst case, where L is given by Equation (8.1). If this constraint is 
binding at optimality with the objective value of the order aah) then the given 


Problem (8.11) can be deduced to be unbounded. 
Adding this constraint along with a slack variable x,,,;, we may rewrite 


Problem (8.11) as follows: 


Minimize {ex: Ax =b,Ix+x,,; =Q,x 20,x,,, 2 0}. 


At this stage, we can homogenize the constraints Ax = b by equivalently 
replacing the right-hand-—side b with b(1x + x,,;)/Q, noting the foregoing final 
equality constraint. However, this will ruin any sparsity structure in A since b is 
usually dense. Alternatively, let us introduce a dummy variable x,,5 along with 
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the constraint x,,,> = 1 to equivalently rewrite the constraints as Ax — bx,» = 9, 
Xn42 = 1, 1X + X41 +%42 =Q+1, plus the nonnegativity restrictions. Using the 
final equality constraint to homogenize the constraint x,,. = 1, we can equiva- 
lently write the problem as follows: 

Minimize {ex : Ax—bx,,5 =0,1x+x,,; —Qx,42 =09, 


1X+ X41 + X42 =(O41),x 20,x,,, 20, x42 2 0}. 


Next, let us use the transformation x; =(Q+l))y;, j = 1,....n + 2, in order to 
obtain unity as the right-hand-side of the final equality constraint above and 
hence derive the following equivalent problem: 

Minimize {cy: Ay —by,, =0,1y + ¥y4) —Q¥ng2 =91Y + V4) + ¥n42 = 


1, 
(8.12) 
y=0,y,,4; 20,442 20}. 


The constraints in Problem (8.12) are now of the form in Problem (8.4). In order 
to satisfy Assumption (Al), let us introduce an artificial variable y,,, into 


Equation (8.12) with objective coefficient M and constraint coefficients such that 
the solution (34,..-,¥n43) =[I/(n+3),...,1/(n+3)]' is feasible. For this to occur, 
the coefficients of each of the resulting homogeneous constraints must add to 
zero, and the coefficient of y,,3 in the final equality constraint must be unity. 
Note that a value of M having a magnitude 2°) exists that ensures that the 
artificial variable y,,,3 will be zero at optimality, provided that Problem (8.12) is 


feasible (see Exercise 8.28). Hence, the input length of Problem (8.12) with the 
artificial variable is still of O(L). This artificial problem is then given as follows: 


Minimize cy+My,,3 


subject to Ay —by,,. -[Al’ -b]y,,3 = 0 
LY + Yast —Wne2 —(24+1-Q)yna3 = 0 (8.13) 
Ly + Ynsi t+ Yn42+In43 = 1 


yj 20, Vj=l...,.7 +3. 


Problem (8.13) is of the form (8.4) in (7 + 3) variables, with [1/(n + 3),...,1/(“ + 
3)]}’ being a feasible solution. Moreover, its size is polynomially related to that of 


Problem (8.11), that is, its input length is of O(Z). 

Finally, consider Assumption (A2). In the following section, we show how 
we can solve a linear program of the form (8.4) that satisfies Assumption (Al), 
but that dispenses with Assumption (A2). Using this technique, Problem (8.11) 
may be solved via Problem (8.13). However, a simple, though computationally 
inadvisable, way of satisfying Assumption (A2) in the earlier derivation is as 
follows. 

By linear programming duality, note that the optimality conditions for 
Problem (8.11) require a solution, if one exists, to the system 


Ax =b, x>0, wA<c, and cx = wb. (8.14) 
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Putting these constraints in a standard form of the type A’x’=b’ and x'>0, the 
foregoing transformations may be used to obtain a formulation having the 
structure of Problem (8.13), where the objective function is simply the 
minimization of the artificial variable y,,3. If the original problem has an 
optimum, then Problem (8.13) has an optimal objective value of zero. Moreover, 
its solution yields both primal and dual optimal solutions to the original Problem 
(8.11). Otherwise, the objective value of problem (8.13) is positive at optimality. 
This will be revealed by the inability of the algorithm to drive the objective value 


below 27’ within the polynomial bound 10n’L’ on the number of iterations as 
established in the following section, where L'=O(L) represents the input length 
of the transformed problem (8.13), and n’ is the number of variables in this 
transformed problem. In this case, the original problem is either infeasible or 


unbounded, and this may be resolved by similarly testing for primal and dual 
feasibility. 


Example 8.3 


To illustrate the use of duality in concert with the foregoing transformations, 
consider the following linear programming problem: 


Maximize 2x; + xX» 


subjectto x) -— XxX, < 2 
x) + 2X < 4 
Xs xX, 2 0. 


The dual to this problem is given as follows: 
Minimize {2w +4w 2: w +wy, 22,-w +2w, 21, w and w, 20}. 
Adding slack variables (x3,x4) and (w3,w4) in the primal and dual problems, 


respectively, we need to determine a solution to the following system in order to 
find a pair of primal and dual optimal solutions: 


X) —X%_ +X = 2, Xj +2x> +4 =4 
Ww + W7 —W3 = 2, —w, + 2w) — wy =1, 
2x) + Xo =2w +4w), and x20, w20. 


Let us now introduce the bounding constraint }°;x; +; w; < Q. (An examination 
of the foregoing system suggests that Q = 15 is sufficiently large.) Adding a 
slack variable s,, this constraint becomes }>);x;+d;w; +s, =Q. Next, we 


introduce a new dummy variable s along with the restriction that s. = 1, and use 
this variable to homogenize the problem constraints. Furthermore, we replace sy = | 
and ¥;x;+;w; +s; =Q with the equivalent constraints ¥j;x,+ L;w;, + 5, - 
Qsy = 0 and 3); x; +; w; +5, +52 =(Q+1). This yields the system: 

xX) —X_ +x3—2s, =0, xX) + 2x7 +x4—487 =0, 


W, + W2 —W3z— 25, =0, —wW +2w2 —wW4—S2 = 9, 
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2x, + Xp -2w, —4w> =0, 
Dix; tL iW; + 5) —Qs2 =0, Dix; tL W; +5, +52 =(Q+D, 
and x20, w20, 5, 20, Sy 20. 
Next, we use a change of variables defined by x; =(Q+)y; for j = 1,..., 4, w,= 
(O+))y4,; for j = 1,....4, 5, =(Q+)yg, and sy =(Q+1)y19. This produces the 
following system: 
¥—Y2+¥3-2ig9 =9, N+ 2y¥2 + ¥4-4Y 0 = 0, 
¥5+¥6—-VY7-2Yj9 =9, = — V5 +2 V6 -—Yg-Vi9 =9, 
2¥ + ¥2 —2 ys —4¥6 = 0, 


9 10 
= ¥; — Dio =0, yy; =I, and yj > 0, Vj =1,...,10. 
i=] i=] 


Finally, introducing the artificial variable y,, with constraint coefficients such 


that the sum of the coefficients in each homogeneous constraint is zero, and 
accommodating y,, in the final equality constraint as well (so that Assumption 


(Al) holds true), we get the following problem: 


Minimize yj, 


subject to Voom Yo * 3 2y0 + Mi = 0 
Wy + 2y, + ye - 40 = 0 

Ve eg Yan = “2g PE oS 

—¥g + 2¥6 - Ye - No + Mi = O 

2y + 2 - 2¥5 - 4¥6 + 3H) = 0 


The problem is now in the required form (8.4), and it satisfies Assumptions (Al) 
and (A2). Using Karmarkar’s algorithm to solve this problem will yield a pair of 
primal and dual optimal solutions to the original linear program. 


8.5 ANALYSIS OF KARMARKAR’S ALGORITHM: CONVERGENCE, 
COMPLEXITY, SLIDING OBJECTIVE METHOD, AND BASIC 
OPTIMAL SOLUTIONS 


In this section we begin with a convergence and complexity analysis of 
Karmarkar’s algorithm applied to Problem (8.4) under Assumptions (Al) and 
(A2). Next, we present a sliding objective function technique that dispenses with 
Assumption (A2) and solves Problem (8.4) in polynomial time under 
Assumption (Al) alone. As before, the polynomial-time rounding routine we 
adopt is a purification scheme that, in fact, yields an optimal basic solution. 
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Using transformation (8.13) of the previous section thus provides the capability 
of obtaining an optimal basic solution to a general linear program (8.11), given 
that one exists. We show that given such a solution, a primal basis B whose 
complementary dual basis is also feasible may be obtained in polynomial time. 
As discussed in Chapter 6, the latter capability is important from the viewpoint of 
deriving insightful duality interpretations in terms of shadow prices and for 
performing useful sensitivity analyses. 


Constructs for the Convergence and Complexity Analysis 


In order to prove the convergence of the algorithm and establish its polynomial— 
time complexity, there are two constructs that we employ. The first is the 
consideration of a relaxation of Problem (8.7), which complements its restriction 
(8.8). The second is the use of a novel function, known as the potential function, 
which measures the progress of the algorithm in a very clever fashion and helps 
establish its polynomial-time complexity. 

To construct the required relaxation of Problem (8.7), consider the n-dimensional 
ball B(yo,R) in the y-space that has its center at yo, coinciding with that of the 


simplex S,,, and that has radius R such that B(yg,R) intersected with {y : ly = 


1} is an (» — 1)}-dimensional ball that circumscribes Sy (see Figure 8.7). Hence, R 
is the distance from the point (1/n,..., I/n)' to any vertex of Sys say, (1, 0,...,0)'. 


This gives R =./(n—-1)/n. Now, consider the problem obtained by adding the 
redundant constraint y € B(yg,R) in Problem (8.7), but relaxing (deleting) the 
nonnegativity restrictions: 


Minimize fey: Py =Posy-Yo)(y-Yo) < R°}. (8.15) 


Again, the feasible region in Problem (8.15) is an (7 — m — 1}dimensional 
ball centered at yo, defined by the intersection of the (n ~ m — 1)}-dimensional 
affine subspace {y: Py = Py} with B(yg,R). Figure 8.7 shows this intersection 


for a case with m = 3 and m = 1. (For an instance with n = 4 and m = 1, a similar 
intersection with respect to the inscribed (shrunk) sphere is shown in Figure 8.5.) 
Hence, the optimal solution y,.y to Problem (8.15) is obtained in a similar 


fashion to Equation (8.9), and is given by 


Re, 


Ynew =YO-T-7W° (8.16) 
le. 


where c p is defined in Equation (8.9b) (see Figure 8.7). We can now obtain an 
estimate for the progress made with respect to the objective function Cy in the 


y-space as follows. Note first of all from Equation (8.9) that indeed Cyew <Cyg 
if c, #9, since C(¥9 —Ynew )=(@ree, ie) =ar|c,|| >0. The last equality here 


follows from the fact that ¢’ =e, +P‘(PP’) 'Pe’ from Equation (8.9b), and so, 
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cc, = leo +cP'(PP') ‘Pc, =le,|. because Pc, = 0 by definition. In fact, 


denoting y* as an optimal solution to Problem (8.7), since Problem (8.8) (solved 
by ynew) 1S a restriction of this problem and Problem (8.15) (solved by Ynew ) is 
a relaxation of this problem, we have 


(0, 0, 1) 


Blyg, RA sly = 13 


Bly, aryOSy 


Figure 8.7. Relaxation over the circumscribed ball. 


— * — 
CYnew SCY SCYnew < CYo- 


Moreover, using this along with Equations (8.16) and (8.9b), we get 


= ~ ere = R _ R_ 
0<€(¥9 —Ynew) S$ C(¥o -Y )S€(Yo —Ynew) =] p =— €(Y9 —Ynew ): 
len] ° ar 


Hence, this asserts that €(yg —y’)<(R/a@r)€(¥q ~Ynew) = (R/ar)\[e(yo — y )- 


C(Ynew — y_)]. Solving for C(Ynew —y’) in terms of C(yo -~y")>0, we obtain 


CWnew ~¥") -) @F_, a 


= — = ; (8.17) 
e(yo—-y ) R (n-1) 
Note that under Assumption (A2), we have ty” = 0, so that 
SMinew gfe os, (8.18) 
CcYo (n-1) 


Equation (8.17) tells us that at any iteration, with respect to the objective 
function cy, the gap to optimality is reduced by 100a@/(n — 1) percent. However, 


¢ =cD, is dependent on the particular iteration k, and moreover, Equation (8.17) 
only guarantees a decrease in the numerator of Problem (8.6). In fact, the 
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fractional objective in Problem (8.6), and hence the original objective function in 
Problem (8.4) may not fall and may actually increase. Fortunately, there is 
another function that preserves strict monotonicity, and hence assures conver- 
gence to optimality. In fact, it assures polynomial-time convergence. This 
function is known as a potential function and is given by 


Sf(xy= x ol = nin(ex) — > In(x;) (8.19a) 
x 


j=l j=l 
where In(-) denotes logarithm to the Naperian (natural) base e. (Taking the anti- 
log of Equation (8.19a), note that we can equivalently employ the multiplicative 


1/ 
potential functionex / [11 = x; | ‘ in lieu of Equation (8.19a). The logarithmic 


transformation is taken here only for mathematical convenience.) 

Observe that while the projective transformation (8.5) does not preserve 
the linearity of a function as seen by the objectives in Equations (8.4) and (8.6), it 
does preserve ratios of linear functions. Indeed, under Equation (8.5), ex (x; = 
cDyy / xy j= ey/xgy >; so that the potential function (8.19a) transforms as 


follows: 


Fly)= sme |- Sn] $Y |= win(y)— ¥ tn(y;)— ¥ In(yg). (8.19) 
Y| jal | Xj j=l j=l 


Let us now measure the decrease in the value of the potential function 
(8.19a), or equivalently in the potential function (8.19b) in the y-space at 
iteration k. Using the potential function (8.19b), we obtain 


F(Ynew )— F(Y0) = rin| Sw —Ynew | 3 In[ "X new) | 
eyo j=l 


since Yoj = 1/n forj = ],..., 4. However, from Equation (8.18), we deduce that 


in| Sa ad |s- & 
Cyo (n-1) (n-1) 


by a property of the In(-) function. Using this equation, we derive 


na n 
f Oniee) NOSE ay In[ AX new) j J: (8.20) 


Consequently, if ay In[“Y(new) j] is sufficiently less than na/(n—-1), 


we will obtain a sufficient drop in the potential function. This will enable us to 
guarantee the desired convergence results. In fact, defining & = na/(m—1) and 


selecting 0 <a < 1 small enough so that /(m—-1)/n@ = Jn/(n—l)a <1, we will 


show that 
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Jaya] <a <1, ly=1, and y>0 (8.21a) 


n a 
0<-¥ In[ny ;] < =a 
jel 2(1-@) 
Observe that y,ew satisfies the conditions in Equation (8.21a) since ly,.y = 1, 


NY new =1| = nar=.{(n—1)/n& <1. Using 
Equation (8.21b) in Equation (8.20), we obtain 

a2 
F(new)- F(¥o) $-@+—2— <2, whe ee) 
2(1-@) -) (n-1) 3 
Therefore, when @ = 1/3, the function F(-)and consequently, the potential 
function f(-), falls by 1/5 every iteration. This means that over k iterations, 


Sf (X4)- F(X) $- WS. But 


implies that 


(8.2 1b) 


Ynew > 9, and from Equation (8.9a), 


(8.22) 


f(xq)- f%) = nin| f > In( my) 
j= 


Xo, 
since X9 = (I/n,...,1/m)‘. Hence, 
n 
nin Fa <¥ In(ny)—*. 
CXo 5 


l/n 
But using the fact that the geometric mean be (xj) is no more than the 


arithmetic mean 1x;/n and that 1x, = 1, we have 


l/n 
n n 
» In(nx;) = nln Il ¥%; < An{1x,]=0. 
ja jal 
This therefore means from the previous inequality that 
in| St] s-4 for all k= 0, 1, 2,... (8.23) 
n 


CX 


Hence, although the objective function value may increase from one 
iteration to the next, a sufficient overall decrease from the original objective 
value at k = 0 is maintained as the iterations progress. In fact, from Equation 


(8.23), when ex; < (exp)e*/™" <27*, we will have CX, < oy, Therefore, when 
k= 10nL, we will have 


ex, < (exe */™" -| 3 cial 2050") s2. 
fal 
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Furthermore, each iteration requires no more than O(n? ) computations, and so 
the algorithm is of polynomial complexity O(n'*L). 
More strongly, observe that 


ae ADZA’ 0 
0 n 


and that the only change in this matrix from one iteration to the next is in the 
elements of the diagonal matrix D,. Accordingly, Karmarkar shows how a slight 


modification of the algorithm, based on updating rather than recomputing an 
appropriate inverse in the gradient projection operation, can be made to run with 


O(n?) effort per iteration. This results in an overall reduced polynomial 


complexity of O(n°°L) for the modified algorithm. 


To conclude this subsection, let us sketch the derivation of Equation 
(8.21a, b). Toward this end, consider the following optimization problem, which 


enables us to bound ate In[ny;] subject to the conditions in Equation (8.2 1a): 


Maximize - In[ny ;]: (ny -1)' (ny -1) (= )#ay =ly2 a (8.24) 
j=l % 


Observe from the first constraint that since its right-hand-side is less than one, 
we must have y > 0 for feasibility in Problem (8.24) (why?). Because the 
objective function is equivalent to minimizing the product [y)v2---y,], it can be 


shown that at optimality in Problem (8.24) (see Exercise 8.30), the first constraint 
is binding. Moreover, it can be shown (see Exercise 8.30) that some 1 < q < (n- 
1) components of an optimal solution are the same and are equal to the value U, 
say, while the remaining (7 — g) components are equal to some common value V, 


where 0 < U< I/n < V< 1. Since ly = land (ny -1)! (ny -1) = [(n-1)/n]a@? must 
be satisfied, we get g¥U + (n -— q)V = 1 and q(nU ~~ 1)? +(n-g)(nV-1)? = [(n 
= 1/n]@?. This gives 

nU= (1-4), nV= 0+6@)) 


a= {2-9 ana pian: (8.25) 
q n (n-q) 


Note that the optimal solution is positive, as required. Hence, we get for some 1 
<q<(n-— 1) that the objective value in Equation (8.24) satisfies 


where 


ay In{ny ;] < —gIn[1 - ,]-(n—g)In[1 + Oy]. (8.26) 
j=l 
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By the well-known second-order Taylor or Mean Value Theorem applied to the 
function In(-), we get for any 0<@<1 that Infl—-O@]= ~0-6" /2A?, where 


(1-6) <A<]. This in turn means that —In[l-6]< 6+67 /2(1—@)?. Similarly, 
—Infl+6]<-O+ 67/2 for any 0<@<1. Using these inequalities in Equation 
(8.26) along with the fact that 0 < 6, <1 for i= 1, 2 and using Equation (8.25), we 


a+ Flom | A, + | 


qa oa a 
20-4)*.  (#-4) 


(a-l) @ [1 -42-9)) 
n 


n 2(1-6,)" 


obtain 


IA 


-¥ Infny] 
j=l 


~2 ~2 
eR ee 
20-4)? 20-@)* 
since 0<@<@<1. This establishes Equation (8.21a, b) and completes our 
derivation. 


< 


Dealing With an Unknown Optimal Objective Value and Obtaining a Basic 
Optimal Solution 


We now describe a method for solving Problem (8.4) under Assumption (Al), but 
without Assumption (A2) that requires the optimal objective value of Problem 
(8.4) to be zero. This method conducts an interval search for the optimal 
objective function value, and is known as a Sliding objective functionmethod. As 
before, we augment this method with a purification scheme that determines an 
optimal basic feasible solution to Problem (8.4). Hence, given any general linear 
program (8.11), we can transform it into the form (8.13) and determine an 
optimal basic feasible solution, if one exists, to the given problem. 

Toward this end, let us first show how we can (polynomially) determine 


whether the optimal value v’, say, for Problem (8.4) is positive or nonpositive. 


Suppose that v* < 0, so that ty* <0, where y’ solves Problem (8.7). Let us pro- 
ceed by assumingthat v" = 0. Now, if CY new <0 at any iteration k, then we know 
that v' < 0 and we stop. Otherwise, if both Cy >0 and Cynew > 0, then noting 


that Equation (8.17) holds irrespective of the sign on v', we have 


a a Qa —* a = a 
stor S14 BE | |e] 
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so that Equation (8.18) continues to hold true. Consequently, we obtain the usual 
decrease of 1/5 in the potential function value as in Equation (8.22) at this 
iteration. If we do not stop before k = 10nL iterations with the indication that v" < 


0, then we will have obtained a feasible solution x, satisfying ex, < re by this 


time. This will then indicate to us that there does exist a feasible solution with a 
nonpositive objective value (why?), and such a solution may be found by using 
the optimal rounding routine of Section 8.4. 


On the other hand, if v" > 0, then denoting B as an optimal basis for 


Problem (8.4), we have v' = N/ |det(B)| for some positive integer NV (why?). 


Hence, we know that v" >1/ |det(B)| oy ag by the definition of Z. Consequently, 


within k = 10nL iterations, we must experience a violation in the promised 
decrease of 1/5 in the potential function value at some iteration, or else we would 


produce an iterate x, with ex, < 2-4, as before. This would then contradict that 
vro2 = Therefore, within 10nZ iterations, we can recognize that v>0. 


Let us now use this technique for recognizing the sign on the optimal 
value of Problem (8.4) in order to solve this problem. To begin with, suppose that 


we have some integer lower and upper bounds ¢ and u, respectively, on v". Note 
that we can take @ as the smallest coefficient c ij (why?) and uv as [exo]. Clearly, 


[2,u] c[-2",2"]. We will now perform a bisection search starting with this 
interval of uncertainty. 

Suppose that we have some current interval of uncertainty, and that z” is 
the midpoint of this interval. Consider the following modification of Problem (8.4): 


Minimize {ex —z’ (1x): Ax = 0,1x = 1,x > 0}. (8.27) 


Note that the objective function value in Problem (8.27) is a constant z” smaller 
than that in Problem (8.4) for all feasible solutions (why?). Hence, the two 
problems are equivalent. Moreover, the objective value in Problem (8.27) is 


nonpositive if and only if z" > v". Furthermore, at the Kth bisection, ze =Ot 


[N(u—£)/2* }, where N is some integer satisfying 0 <N< 2* . Also, the current 


9k 


interval of uncertainty is of length (~—2£)/ . As long as the interval of 


uncertainty is of length 2 ae we have = d/284 a) Baan ae ae 
(u—0)27" < ee, that is, we have K < 3L + 2. This means that as long as the 
interval of uncertainty is of length = 27+ we have Nand 2% bounded above by 


pete: Therefore, after integerizing the objective in Problem (8.27) by mul- 


tiplying with 2X | we still have the size of the integer input length in Problem 
(8.27) of order O(L). Consequently, in O(#Z) iterations, we can recognize the 
sign on the objective function in Problem (8.27) as before, and therefore 
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ascertain whether z* > v" or z” <v". This enables us to halve the current interval 
of uncertainty. (The minor details of this argument follow readily and are left to 
the reader in Exercise 8.35. Note that in practice, we can use the best-known 
solution generated at any bisection stage to further reduce the interval of 
uncertainty.) In this fashion, we can continue bisecting the interval of uncertainty 


until it is reduced to length less than 2+. This happens after at most 3Z + 2 
bisections. 
Now, denote the final interval of uncertainty as [f,u]. By applying the 


optimal rounding routine following the solution of Problem (8.27) with z* = u if 
necessary, we can assume that we also have a solution x to Problem (8.4), with 
objective value Z <u. Starting with the solution x, we can use the purification 
scheme to obtain in polynomial time a revised solution x, which is an extreme 
point of Problem (8.4) having an objective value ex <Z. This resulting basic 
feasible solution x is indeed optimal to Problem (8.4). If this were not so, then 
there would exist another basic feasible solution x of objective value cx < cx. 
Denoting B, and B, as the bases corresponding to x and x, respectively, we 


would then have (with obvious notation) 0 <ex—ex = N,/D,-N>/D> where 


D, =|det(B;)|, i = 1, 2, and where N, and N> are integers. Hence, this would 
yield ex—cX >1/D,D, >2->” by the definition of L, leading to ex < cx-27%, 


This contradicts that (vu -¢) < 27°" and that Z and v* €[¢,u]. 

This method provides a technique for obtaining an optimal extreme point 
solution for a general linear program of the form (8.11). After converting it to the 
equivalent form (8.13), we can use this procedure and determine a basic optimal 
solution to Problem (8.13). Of course, if y,,3 >0 at optimality, then Problem 


(8.11) is infeasible. Furthermore, if y,,; = 0 at optimality, being nonbasic with a 


negative “‘z je; Value, then Problem (8.11) is unbounded (why?). Otherwise, 


¥n+3 =9, Vyr1 29 is basic, and y,,> =1/(Q+1) is basic at an optimal extreme 


point of Problem (8.13). Therefore, the corresponding solution to Problem (8.11) 
obtained via the transformation x; =(Q+1)y;,j = 1,.... 2, gives an optimal basic 


solution to Problem (8.11). Furthermore, using the corresponding optimal basis 
B, we can solve the system WB = cp (in the usual notation) in polynomial time to 


obtain the associated complementary dual basic solution. If this is feasible, then 
it is also optimal for the dual problem. However, in the presence of degeneracy, 
some degenerate simplex pivots may be required in order to obtain an optimal 
basis B whose complementary dual basis is also feasible. Although this latter step 
is not of polynomial complexity, such a basis B can indeed be obtained in 
polynomial time as follows: 

Suppose that in addition to the foregoing primal vertex, an optimal dual 
vertex with an associated basis is also available (via the use of a similar 
polynomial-time scheme, if necessary). Denote by vj, J = 1,...0, the slack 
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variables in the constraints for the dual to Problem (8.11). Let Vj, J = 1,...5”, be 


the values of these variables at the given optimal dual vertex. Define J = {je€ 


{1,...,.2}: ¥; = 0}. Note that the set J includes the indices of the m nonbasic vy ie 


variables and those of any degenerate basic v j variables. Hence, the columns of 


x; for j¢J have rank m. Moreover, the latter columns include those for the 


currently positive x,—variables, which are themselves linearly independent. 


Hence, we can construct a primal basis B by Gaussian elimination in polynomial 
time, using the columns of the x j variables for 7 € J, while including those for 
the positive x ;—variables. This basis B represents the given primal vertex (why?). 
Furthermore, the given dual solution is the (unique) complementary dual basic 
solution corresponding to B, and so the basis B satisfies the desired property. 


Example 8.4 


Consider the following linear programming problem: 


Minimize -Sx, — 6x, - 4x; - 4x4 


subjectto x — XxX) + 2x3 — 2x5 = 0 
xy + 2x9 + + xXg 7 4x5 = 0 

Xt Xp + X + Xy + X = 1 

x > 0. 


Notice that if we add 4(x, +x2 +23 +X4+xXs5) to the objective function, we obtain 
the problem of Example 8.2. Hence, the optimal objective value for this example is 
v" =-4, attained at (0,2/5,2/5,0,1/5)’, as in Example 8.2. 


Suppose that we assume the optimal objective value to be z* = -3. Sub- 
tracting -3(> ;x,;) from the objective function modifies this function to: 


Minimize — 2x, —3x2 — x3 —X4 +3xs. 


Notice that since z” >v", the optimal value with respect to the foregoing 
modified objective is negative (in fact, it is equal to —1). Applying Karmarkar’s 
algorithm while assuming this value to be zero results in an objective value of 
-0.8 corresponding to the starting solution itself. Hence, we immediately 


recognize that v <z* =—3. 
On the other hand, suppose that we assume the optimal objective value to 


be z’ =—5. Subtracting —5(> j%;) from the objective function modifies this 
function to: 

Minimize — x7 + x3 +X4 +5x5. 
Since z” <v’, the foregoing modified objective has a positive optimal value (in 
fact, it is equal to +1). Applying Karmarkar’s algorithm and assuming this value 


to be zero gives at the very first iteration an improvement in the potential 
function of the amount 
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Cc n 
F(Ynew)-F (yo) =” hl Ynew | x In[7V(new);] 
cYo j=l 


0.221606 


= (5)! 
©) n| 0.22973 


(The reader may verify these values by performing one iteration of Karmarkar’s 
algorithm.) Since the guaranteed descent of 0.2 in the potential function value is 


- In(0.9505476) = —0.12932. 


not obtained at this iteration, we recognize that vy" >z* =~5. The bisection search 
uses such a technique to iteratively reduce the interval of uncertainty to a 
sufficiently small length. 


Example 8.5 


Consider the problem of Example 8.4. Let us illustrate the purification procedure 
by arbitrarily applying it to the starting solution xq = (0.2, 0.2, 0.2, 0.2, 02)! 
itself. Here, the three equality constraints alone are binding. A direction d lying 
in the null space of these constraints should satisfy d; — d, + 2d3 — 2d; = 0, 
d, +2d,+d4—4d5=0, and d| +d, +d3;+d,+ds5 =0. Solving this system for 
d,, 43, and d4, say, we obtain d; =—dy —8ds5, d3 = d,+5ds5, and d4, = —d + 
12d. Hence, for example, arbitrarily putting d) =1 and ds = 0 gives a direction 


d =(-l, 1, 1, -1, 0) that keeps the binding constraints tight. Note that ed = —1, 
and so this direction is improving. Moving along this direction, we are blocked 
by both x, and x4 dropping to zero simultaneously at a step length of 0.2 


(why?). This results in the new (improved) feasible solution (0.2, 0.2, 0.2, 0.2, 
0.2)’ + 0.2(-1, 1, 1, -1, 0)’ = (0, 0.4, 0.4, 0, 0.2)’. It may be verified that the 


three equality constraints in the problem along with x, = 0 and x4 = 0, which 


are binding at the current solution are linearly independent. Hence, we have 
reached an extreme point solution. For this problem, we have coincidentally 
attained optimality by purifying the starting solution itself. Since this optimum is 
nondegenerate, the associated complementary dual basic solution is also optimal. 

As far as computational experience with the basic algorithm and the 
transformation (8.13) is concerned, it has been shown to yield very competitive 


results with respect to the simplex algorithm, particularly for large (m+ n> 10*) 
and sparse problems. (Many of the variants of Karmarkar’s algorithm discussed 
in Section 8.6 and mentioned in the Notes and References section have been 
shown to be substantially faster than the simplex algorithm for such problems.) It 
may be noted that Karmarkar’s algorithm and its variants typically converge 
within about 50 iterations (almost independently of problem size!), making very 
rapid initial improvements in objective value in relation to their tail—-end 
convergence rate. This implies the following two noteworthy points. 

First, the algorithmic efficiency can be improved by reducing the effort 
per iteration. Note that the major task in an iteration of the algorithm is to 


compute the projection c p in Equation (8.9b). This step also requires a very 
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careful implementation in order to avoid numerical errors from a rapid loss in 
feasibility of the iterates. (See the Notes and References section for a variant that 
permits approximate projections.) Recall that c,, in Equation (8.9b) is given by 


2 
c’ — P'w, where W solves the least-squares problem le" - Pw| over we R™*, 


A factored-form implementation is recommended to control the numerical 
accuracy while solving this problem (see Exercise 8.40). Hence, the effort for 
computing c, can be reduced by solving this associated least-squares problem 


more efficiently. Alternatively, an appropriate technique may be used for directly 
computing (PpP’)! in Equation (8.9b), which exploits the fact that the only 


change from one iteration to the next in computing c,, is in the diagonal matrix 
D,. 

Second, it also has been suggested that one use Karmarkar’s algorithm for 
some initial iterations, “purify” the available solution to an extreme point having 
at least as good an objective value, and then switch over to the simplex algorithm 
in order to complete the optimization process. This has yielded some 
encouraging results. Another suggestion that appears worthwhile is to drop 
variables that are approaching zero from the problem as the iterations progress, 
and hence reduce the dimension of the problem as well as avoid the possible ill- 


conditioning of the matrix PP’, which is associated with its determinant getting 
too close to zero. This is done at the expense of somehow later resurrecting some 
deleted variables if necessary. Strategies of this type may be used to enhance the 
computational performance of the algorithm. 


8.6 AFFINE SCALING, PRIMAL-DUAL PATH FOLLOWING, 
AND PREDICTOR-CORRECTOR VARIANTS OF INTERIOR 
POINT METHODS 


Inspired by the concept of Karmarkar’s algorithm of deriving an efficient proce- 
dure for solving linear programming problems by traversing a trajectory that 
seeks a pathway toward optimality through the relative interior of the feasible 
region, there followed a flurry of activity at devising alternative mechanisms for 
generating such trajectories that might lead to computationally more effective 
procedures. A vast body of literature has accumulated since then (see the Notes 
and References section for some key survey articles), and collectively, the 
different variants of such procedures are referred to as interior point methods. 
Indeed, many of these methods are rooted in some well-known classic nonlinear 
programming algorithms that predate Karmarkar’s method by about two decades. 
Some of these variants as specialized for linear programming problems are not 
known to be polynomially bounded, whereas others have been shown to have an 


improved polynomial complexity of O(n L) as compared with the O(n? >L) 
procedure proposed by Karmarkar. More importantly, suitable implementations 
of these methods, which frequently ignore the theoretical restriction on step— 


lengths that guarantee polynomial-time behavior in favor of computational 
efficiency, have proven to be very effective and preferred alternatives over the 
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simplex method. This is particularly so for large-scale (m + n > 10* or so) and 
sparse linear programs that do not possess any particular special structures 
(which are typically exploitable by the simplex method). We briefly discuss in 
this section some of the more effective variants of these interior point methods 
that have been popularly adopted by modern-day commercial software, and we 
refer the reader to the Notes and References section for further reading on these 
methods and their convergence characteristics. 


Affine Scaling Methods 


Soon after Karmarkar proposed his novel approach, several researchers 
questioned the use of the curious form (8.4) of linear programs, as opposed to the 
standard form of primal and dual problems given below, where A is an m x n 
matrix of rank m: 


P: Minimize ex D: Maximize wb 
subjectto Ax=b subjectto wA+v=c (8.28) 
x20. v = 0,w unrestricted. 


Additionally, they also questioned the use of the projective nonlinear 
transformation (8.5), in contrast with the related affine transformation: 


y=D,'x, ie, x=D,y, (8.29) 


where as before, D, = diag {x,1,...,.%%,}, given a feasible interior point solution 
x; > 0. This gave rise to the class of affine scaling methods, which actually 
rediscovered an identical procedure proposed previously in the Soviet literature 
by I. Dikin in 1967. Under the transformation (8.29) at any iteration k based on 
the current interior feasible solution x;, Problem P is equivalently given by the 
following problem in the y—space: 

Minimize cD,y 

subjectto AD;,y b (8.30) 

y 0. 

Defining the gradient of the objective function in (8.30) as ¢ =cD,, and follow- 


IV Il 


ing the same derivation as for Equation (8.9b) with AD, playing the role of P in 


this context, we can derive the projection of ¢’ onto the null space of the equality 
constraints in (8.30) as follows: 


¢, =[1-(AD,)‘[ADZA‘T '(AD, Ie. (8.31) 
If c,, = 0, then the current solution y; = Dz! x; =1' is optimal for (8.30), and 


correspondingly, the solution x, (and indeed, any feasible solution to P) is 


optimal for P (why?). Otherwise, we take a step size A along the negative pro- 
jected gradient direction to obtain 


Yeu =Yx —ACp. (8.32a) 
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Using (8.29), this translates to the following motion in the x-space upon 
premultiplying (8.32a) throughout by D,: 


Xxa1 =X, —ADc, = xy + Ady, (8.32b) 


say, where d, =—-D,c,,. Note that the maximum value 4,,, of 4 that would 


maintain feasibility of x;,,, is given by the usual “minimum ratio test” as: 


a XK 
| inn dy < ot (8.33) 
ij 


Assuming that the feasible region is bounded (by using an artificial bounding 
constraint if necessary), we have A,,,, <0. However, in order to maintain the 


positivity of the iterates, we adopt a step—length that is short of the maximum 
step length, i.e., we take 


A = Ohinax » where 0<a<l. (8.34) 


(In practice, @ is selected to lie in the interval 0.95-0.99.) This completes an 
iteration. Incrementing & by one, the foregoing process is repeated. It can be 


shown that if this process generates a sequence {lke ol} — 0, then the corresponding 


sequence of iterates {x x} approaches x", an optimal solution to P, with the sequence 
wi, = (AD? A‘)! (AD, ye’ approaching an optimal dual solution. (Hence, a prac- 
tical termination criterion is to stop the process whenever le | becomes suffi- 


ciently small.) If both P and D are nondegenerate, this convergence can be shown 
to hold true for any 0 < a < 1. However, under degeneracy, it has been exhibited 
through an example that the procedure can fail to converge to an optimum 
whenever a > 0.995, whereas one can prove in general that convergence (even 
under degeneracy) holds true provided a < 2/3. Note that when convergence does 
occur, we can optimally round the solution to an exact solution in a finite number 


of iterations once the optimality gap becomes smaller than 7 is , as before. 
However, this convergence process is not known to be polynomially bounded. In 
fact, it is suspected that by making a < 1 sufficiently large, the procedure can be 
contrived to follow a path close to the boundary that approximately parallels the 
simplex path described for the Klee-Minty example in Section 8.2, thereby 
portending an exponential worst-case complexity behavior. Nonetheless, this 
method is quite effective in practice. Finally, we comment that the foregoing 
procedure has been applied to P and is therefore called the primal affine scaling 
method; similar approaches adopted for D, or for both P and D simultaneously 
give rise to the respective classes of dual affine scaling, and primal—dual affine 
scaling methods. 
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Primal-Dual Path—Following Methods 


Consider the pair of primal and dual linear programs in Equation (8.28) where 
the feasible region for P is nonempty and bounded (perhaps artificially so), and 
let us examine the following Barrier Problem BP, which is parametrized by a 
barrier parameter p> 0: 


BP: Minimize | B(x) = ex— pS In(x;): Ax = | (8.35) 
j=l 


Observe that the nonnegativity constraints in Problem P of Equation (8.28) have 
been incorporated into the objective function in Problem (8.35) along with a 
“barrier term” that approaches oo if any x ;—value approaches zero (from the positive 


side). It is well known from nonlinear programming (see the Notes and Refer- 
ences section) that given any desired level of accuracy ¢>0, there exists a 
#t>0O small enough such that the corresponding optimal solution to Problem 
(8.35) is positive and has an objective value in Problem (8.28) within ¢ of 
optimality. In fact, by the strict convexity of the objective function in Problem 
(8.35) and the stated assumptions on P, for every 4 > 0, there exists a unique 


optimum x,,, Say, to (8.35), where x,, approaches an optimum for P as > 0*. 


In fact, similar to the case of linear programs, the following Karush-Kuhn-Tucker 

(KKT) conditions for (8.35) are both necessary and sufficient for optimality, 

where the right—hand-side of Equation (8.36b) is the gradient of the objective 
function in Problem (8.35): 

Ax=b (8.36a) 

wA =c-— p{1/x,...,1/x,]- (8.36b) 


To imitate the KKT conditions for P, and noting the form of D in Problem (8.28), 
let us substitute v = 4{1/x,...,1/x,] in Equation (8.36b). This yields the so— 
called perturbed KKT conditions (which are equivalent to Equation (8.36), but 
are specially designated because they admit algorithmic steps and related 
performance characteristics that would be quite different from applying similar 
techniques directly to Equation (8.36)): 


Ax=b (8.37a) 
wAt+v=c (8.37b) 
vjxp =H, Vi = 1,...n. (8.37c) 


Note that given any yw > 0, and the corresponding unique optimum x,, > 0 to 


Problem BP, Equation (8.37c) yields an associated unique value of v y > 9, and 


Equation (8.37b) then yields a unique value for w yz since A has full row rank 
(note that a solution exists since these are necessary optimality conditions). Let 


us denote the triplet (x,w,v) by &, and accordingly, let Su =(x,>Wy>Vy)- 


(Note that throughout, for notational convenience, x is a column vector whereas 
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w and v are row vectors). The trajectory ¢,,, generated as a function of yw as 


i> 0°, is called the central path (motivated by the interiority of the solutions 


thus produced). Observe that from Equations (8.37a)-(8.37c), we have that the 
optimality gap ex — wb is given by 


n 
ex—wb = vx+ WAx—WAX= D0 vx; = ny. (8.38) 
j=l 
Hence, as 4 > 0*, the sequence of primal and dual feasible solutions x y and 
(Ww ,,V,,) tespectively approach optimal solutions to P and D. 

However, rather than follow the central path precisely (which would be 
computationally demanding), it turns out that it is sufficient to adopt a path that is 
close enough to the central path in a well-defined sense, in order to achieve 
(polynomial-time) convergence. More specifically, at any iteration k, suppose 
that we have a current solution ¢, = (x,,W,,¥,), which is close enough to ¢,, 


for some current value of 4, > 0 in that: 


Ax, =b, x, >0,w,A+v, =¢, v; > 0, (wy, is unrestricted),  (8.39a) 


n n 
| 2 (o4jay - Hy SO, while 2 MigXiy =e (8.390) 
j= j= 


where 0 < 6 <0.5 is a constant. Note that whereas Equation (8.37c) might hold 
true only approximately in the sense of Equation (8.39b), the second condition in 
(8.39b) asserts that the sum of the equations in Equation (8.37c) should hold true 
exactly. In particular then, by Equation (8.38), the duality gap cx, —w,b equals 


and 


ni, We now reduce the value of 4 to 
Mkt = Pee (8.40) 


where 0 < £ < 1, and accordingly, we derive a direction of motion dz = 


(d,,d,,,d,,) in the primal—dual space based on an attempt to satisfy Equation 
(8.37) for this revised value of 4 = y,,). Specifically, let us therefore examine 
the system 
A(x, +d,)=b 
(w, +d,)A+(v, +d,)=c 
(Vy ty, xy + dy )= Hest VW= | er 
Noting Equation (8.39a), this reduces to the system 
Ad, =0 (8.41a) 
d,A+d,=0 (8.41b) 


Complexity of the Simplex Algorithm and Polynomial-Time Algorithms 433 


vga, + xyjdy, = Hin Vata Fy x, Vj =1,...,m. (8.41c) 


Observe that the system (8.41) is linear except for the quadratic product term 
dy. d x; in Equation (8.41c). Moreover, if we neglect this product term, then this 


system yields a unique solution dz =(d,,d,,,d,) (see Exercise 8.53). Indeed, 


this direction turns out to be equivalent to that obtained by applying a single step 
of Newton’s classic optimization method to the perturbed KKT system given in 
Equation (8.37). Adopting a unit step size along this (Newton) direction, we 
obtain the new iterate 


Sk+t = Sete. (8.42) 


It can now be shown, noting Equations (8.39) and (8.40), that if we select 


re) 
=]-—~, where 6 = 0 =0.35, 8.43 
B af (8.43) 


say, then the new iterate &,,, thus obtained would be close enough to Sea on 


the central path in the sense defined by Equation (8.39) (see Exercise 8.54). 
Hence, in particular, the optimality gap would be given by Equation (8.38). This 
means that for each iteration k, we would have that the optimality gap satisfies 
(noting (8.40) and assuming that voxq = mfp to start): 


exp — Wb = nLly = Melo B* = voxof* <|vol|xo|B*<2°*B*, (8.44) 


where it can be shown that ||xq||<2” and |v <2” (see Exercise 8.10). There- 


ree thereby permitting an optimal polynomial- 


fore, this gap would be lesser than 2 
time rounding as in Section 8.4, whenever are Bp" eg ae eS AL In(2)/ 
[-In(£)]. But from (8.43) and a property of the In(-) function, we have 
re) o 
—In(f) = -In(i -—=) = —. 
Vn Vn 


Therefore, by the iteration k > 4LZ In(2)/[6/ Vn], that is, in O(./nL) iterations, we 


would have the optimality gap being lesser than 24. Similar to Karmarkar’s 
algorithm, the effort per iteration is O(n? ) (see Exercise 8.55), leading to an 


overall Ow D) algorithm. 


Note that because the value of # is very close to unity, especially as n 
increases, the decay of s+-values as controlled by Equation (8.40) can be very 
slow, leading to a painfully slow convergence process. Hence, a somewhat more 
aggressive strategy of decreasing wis adopted in practice, thereby sacrificing the 
provable polynomial-time behavior in exchange for a highly improved empirical 
computational performance. For example, one strategy that has been successfully 
used is to let 
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Mya = 1,/n ifn < 5000, and 4,4; =4,/Vn ifn>5000. — (8.45) 


In closing, we comment here that an application of an affine scaling strategy to 
the barrier problem BP defined in Equation (8.35), instead of to the original 
underlying linear program P, recovers the standard affine scaling method in the 
limit as 4 +0. However, by holding 4 > 0 but sufficiently small to enable an 


optimal rounding, the convergence of such an affine scaling process can be 
shown to hold for all values of the step—length parameter 0 < a < 1 analogous to 
Equation (8.34) (see Exercise 8.43), regardless of degeneracy. 


Predictor—Corrector Algorithms 


One of the most computationally effective interior point variants is a predictor— 
corrector approach, as applied to the foregoing primal—dual path—following 
algorithm. The basic idea behind this technique originates from the successive 
approximation methods implemented for the numerical solution of differential 
equations. Essentially, this method adopts steps along two successive directions 
at each iteration. The first is a predictor step that adopts a direction based on the 
system (8.41) under the ideal scenario of 4,,; =0, but again neglecting the 


second—order term d,, x)? Vj = 1,...,m. Let di: be this direction obtained. A 


tentative revised iterate €j,; is computed as j,, =¢, +Ads, where the step- 


length 4 is taken close to the maximum step length A,,,, that would maintain 


positivity of the x— and v—variables, similar to the mechanism of Equation (8.34). 
(Here, we can take @ = 0.95-0.99, say.) Then a revised value 44,4, of the 


parameter yz is determined via a scheme of the type of Equation (8.45) or based 
on some suitable function of the optimality gap or complementary slackness 
violation (see Equation (8.38) and Exercise 8.57) at & and at é,,. Using this 


value of j4,,; in Equation (8.41), but this time replacing the quadratic product 
term in Equation (8.41c) by the estimate qd, ty? Vj = 1,...,#, as determined by 
di‘, in lieu of simply neglecting this nonlinear term, a revised direction de is 
derived. (Note that because of the similarity of the systems that determine dz 
and dz, factorizations developed to solve for the former direction can be 
reutilized to obtain the latter direction.) The revised direction d é thus 


determined is then used to take a step from € to give the new iterate &,,,. This 


correction toward the central path is known as a corrector step. Observe that the 
system (8.41) could be solved repeatedly in this manner by using the most recent 
direction components to estimate the quadratic term on the right—hand-side in 
Equation (8.41c), in order to obtain a more accurate estimate &,,, corresponding 
to the parameter jz, ,,;. Numerical results suggest that this might be a promising 


alternative to using a single corrector step. 


Complexity of the Simplex Algorithm and Polynomial-Time Algorithms 435 


EXERCISES 


[8.1] Solve the problem to minimize 3x, + 2x2, subject to x) + 2x2 22 and x, 20, 
X 20, using Karmarkar’s algorithm by equivalently transforming the problem into 


the form (8.4) that satisfies Assumptions (Al) and (A2). Plot the iterates you 
generate in the (x),x> ) space. 


[8.2] Consider the problem to minimize 3x, +2x. subject to x +2x,—x3; =2, 
X +X +%3+x4 =4, x > 0. Transform this equivalently into the form (8.4) as in 
Equation (8.13) that satisfies Assumption (Al), but not necessarily Assumption 
(A2). Use Karmarkar’s sliding objective function method described in Section 8.5 
to solve the resulting problem. 

[8.3] Provide the details of an approach that uses a polynomial-time algorithm for 
the linear programming decision problem in order to polynomially solve the linear 
programming optimization problem. What is the complexity of the latter in terms 
of the complexity of the former? 


[8.4] With respect to the problem of Equation (8.3), show that the (z j 7¢;)-values 


corresponding to the basis with basic variables (5),5},...,5,-],¥,) are all 
nonnegative. 


[8.5] Demonstrate that the transformation (8.2) reduces the Klee—Minty problem 
to the form in Problem (8.3). 


[8.6] Following the arguments in Section 8.2, give a detailed proof by induction 


that for Problem (8.3) in n—dimensions there exists a simplex path of length 2” —1 


from the origin to the optimum using the most negative z;—c j entering rule 


(Dantzig’s Rule). 

[8.7] Plot the (original) Klee—Minty polytope for n = 2 and n = 3. What is the 
relationship between these polytopes and the ones shown in Figure 8.1? Analogous 
to the statement of Exercise 8.6, what precise complexity assertion can you make 
with respect to the original Klee—Minty problem? 


[8.8] Consider the linear programming problem to: 


n 2 
Maximize > 10"! x; 
j=l 


il , 
subject to 20h, jes s100" fori=1,....7 
j=l 


xj 20 for j =1,...,7. 


a. Demonstrate graphically for n = 2 and n = 3 that using (Dantzig’s) 


most negative z;—c; entering rule, the simplex method can go 


through 2” -1 pivots, starting at the origin. 
b. Provide a proof by induction for the statement in Part (a) for general 
n. 
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[8.9] Given a square matrix B with components B,, , i= 1,..., m, 7 = 1,..., m, show 
by induction that |det Bi < TT? 5-1 (dl +|B,)). (Hint: Use the column (row)-wise 
expansion formula for det B.) 

[8.10] In the usual notation, let x = (Xz,Xj,) be a basic feasible solution of the 


system Ax = b, x > 0, where A is m x 1 of rank m and where the data is all integer. 


Show that the Ath component Xp, of Xp satisfies xp, < 2! /(mn), where 


| 


L=[1+logm+logn+Z¥|1+log(1+ ay 
ij 


+¥[1+ log +|4;){] 1. 
i 
Here, aj and b; are the components of A and b, respectively. Hence, show that 
|x| <2” /n. 


[8.11] For the linear programming problem to minimize cx subject to Ax = b, x > 
0, where A is m x n and where the data is all integer, define L as in Equation (8.1). 
Using Exercise 8.10, and assuming the linear program to be feasible and bounded, 


show that the optimal objective value lies in the interval [aoe ; ae ]. 


The following exercises 8.12-8.17 relate to Khachian’s algorithm for 
determining in polynomial time if there exists an x belonging to the set 


S= {x: Gx<g} 
or to conclude that S =@, where G is r x q, g is r x 1, with r and q > 2, and 
where the data is all integer: 


[8.12] Define 
L=[1+logr+logg+ DX{1+log(+|Gy) | 
ij 
+Z[1+ ogi +|g;)| | 
I 
and let Eq = {x:||x|| <2}. Show that if S + ©, then 


vol[ Ey AS]= 2" 


where vol(1) denotes the g—dimensional volume. (Hint: The volume of a q— 


dimensional simplex with vertices vj,...,.¥,,; is given by 


| (Ce ern | 
det 
: E M2, vs on 
[8.13] (Khachian’s Main Step 1) Starting with k = 0, suppose that at iteration 4, 
the ellipsoid £, defined as follows is given: 


Ey, = {x:(x—x,)'Qg'(x-x;,) <}} 


(1/q!) 
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where x, is the center of the ellipsoid and Q,; is aq x q symmetric positive 
definite matrix (that is, y'Q.y >0 for all y#90). In particular, det(Q,)¥ 90. 
Given E,, if x, €S, then halt. Otherwise, let the most violated constraint 
defining S be G,x<g,. Hence, G,x,; 2g,. Geometrically, translate the 
hyperplane G,x = g,, parallel to itself in the (feasible) direction of -Gi until it 
becomes tangential to the ellipsoid £, at the point y, (see Figure 8.8). Show 
that 

-Q,G;, 


¥G,Q,G;, 


Next, compute Y;, as the point on the hyperplane G,x = g, obtained by moving 


Yr = XE +d;, where d; = 


from x, in the direction d;. Hence, Y, =x, +A,d, for some step length A,. 
Show that A, =(g,—-G,x;,)/(G,d,) and that A, 20. Also, show that if A, 21, 
then S = © and the procedure can terminate. 

[8.14] (Khachian’s Main Step 2) Assuming that A, <1 in Exercise 8.13, let x,,1 
be the point on the line segment [Y;,,y,] that divides this in the ratio 1:¢. Hence, 


q l (1+ Aq) 
=| —— |Y, +] — =x, +——"—d 
oe (4) ‘ Gaz Peg, 


Define the new ellipsoid E,4) = {x:(x—X,41) Qi, (x-X;4,) <1} with center at 
X,4, and with Q;,, given by the following expression: 


2 2 
1-42) 2(1+gA;,) 
Q.1 =| ¢ . C nee aa | 
(q* -1) (q+I(A, +1) 
Show that: 
a.  Qy4; is symmetric and positive definite (given that so is Q;). 


The point y, lies on the boundary of F;,,;. 
c. The hyperplane G,x = G, y, is tangential to E,,, at yx. 


d. Ey A{x:G,x< gy} C Ey, where E, = {x:(x—x,)'Qy! (x-x,) < Bh. 
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G,x =Gyy, 


G,x= gy 


<— halfspaceG,x < g, 


Ex 


Figure 8.8. Geometry of the main step of the ellipsoid algorithm. 


Inductively show that Ey NS CE, VS C Ey for all k = 1, 2,..., that is, each of 
the ellipsoids E, , E> ,... contains at least that portion of S that was captured by Ep. 


[8.15] Let V, and V2 be two vectors such that V{ V2 > 0. Let @be a scalar. Using 
suitable row operations, show that 


det/1-0V,V5 IV{V2 | = (1-8). 


[8.16] Using the fact that the volume of the ellipsoid E, equals Vo ./det(Q,), 


where Vo is the volume of the unit sphere in R17, use Exercise 8.15 to show that 
vol(E;,) < e 2149) vol(Eo) < e K/2(1+q) qqtl) 


[8.17] Using Exercises 8.12-8.16, conclude that if S #@, then by 6(q¢ +1)? L 
iterations, the algorithm will find a solution in S. Hence, if no solution is found in 
6(q+1)7L iterations, then S =@. Therefore, show that the overall complexity of 
the algorithm is O(q'L). 
[8.18] Consider the open system of inequalities Gx < g that defines the set 

S= {x = (x1 ,X2):- —X2< —1, x; < 1}. 
Show that with Z = 2, the assertion of Exercise 8.12 holds true. Use Khachian’s 


algorithm with L = 2 to find a solution to this system. Illustrate the progress of 
the algorithm graphically. 


[8.19] Consider the open inequality system x; —x2 >1, 2x, +3x2 >6, x, >0, 
X_ >0. Show that with ZL = 2, the assertion of Exercise 8.12 holds true. Use 


Khachian’s algorithm with L = 2 in order to find a solution to this system. Illustrate 
the progress of the algorithm graphically. 
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[8.20] In order to be able to use Khachian’s algorithm for solving linear 
programming problems, one needs to be able to determine if the closed system of 
inequalities Gx < g has a solution, where G is r x g, with r and g > 2, and G and g 
have all integer components. Toward this end, define S = {x : Gx < g} and 
consider the following open inequality system, where 1 is a row vector of r ones, 
and where L is defined in Exercise 8.12: 


Gx<g+(2/)I', — thatis, (24)Gx <(2/)g+1', 


Use Farkas’ Lemma (see Chapter 5) to show that Sis nonempty if and only if 
this open inequality system has a solution. Show that the complexity of deter- 


mining whether or not Sis empty by Khachian’s algorithm is O(rg>L). 
[8.21] (Optimal Rounding Routine) Suppose that we are given a solution x to the 


open inequality system Gx < g + Vg )1', where L is defined as in Exercise 8.12. 


Suppose that we are to determine from that a solution to the system Gx < g. Of 
course, if Gx < g, then we are done. Otherwise, define s;(x) = g; —G;x as the 


slack in the ith constraint G;x < g; corresponding to a given solution x. Note that 


we have s;(x)<0 for some ‘e {1,..., r}, but that s;(x) > 2 for each i = 1...., 7. 
(This evidently means that only a small adjustment in x may be required to obtain 
s;(x)20 for all i = 1,..., 7 hence, the name of this routine.) Consider the 
operations of the following two steps: 

Step 1. First, refine xX to a solution Xyey, which is such that (i) 
5;(Xpew ) 2 min {0,s;(x)}, and (ii) every row of G can be written as a linear 
combination of the rows with index set V = {i : s;(Xjey ) S$ 0}. Show how this can 
be achieved in polynomial time. (Hint: If x does not satisfy these conditions, 
select a row G, of G, with k ¢V, which is not spanned by G,, ie V, and find a 
solution d(# 0) to the system of equations G;d =0 for ie V, Gd =1. Consider 
the ray x = x+Ad, 420, and take a suitable step along this ray. Repeat.) 

Step 2. Given Xyey from Step 1 along with the set V, find a maximal 
linearly independent set of rows from the set G;, ic V. Let these rows be G;, 
i¢éJ CV. Then show that any solution to the system G;x = g; for ie J, yields 
the desired solution to the system Gx < g. 

Derive a polynomial complexity bound for this routine. 


[8.22] This exercise describes an alternative approach to determining a solution to 
a closed system of inequalities via Khachian’s algorithm. Consider the closed 
system of inequalities Gx < g, where G is r x g. Suppose that we have a solution x 
satisfying G;x<g; for i = l,..., p, where p < r. Examine the open system of 


inequalities G,;x < g;, for i = 1,..., p + 1, and suppose that this system is empty. 
Show that G ,,;x = ¥ p41 for all feasible solutions to Gx < g. By solving for some 


variable in this equation in terms of the others, show how the system Gx < g can be 
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reduced to an equivalent one with one fewer variable and constraint. How can you 
repeatedly use this process along with Khachian’s algorithm to determine in 
polynomial time whether or not a closed system Gx < g (with integer data) has a 
feasible solution? What is the complexity of this procedure? 

[8.23] Consider solving the following linear program in canonical form: Minimize 
ex, subject to Ax > b, x > 0, where A is m x n. Show that solving this problem is 
equivalent to determining a solution to a certain closed system of inequalities. How 
would you decide if the given linear program is infeasible or unbounded, given that 
the latter inequality system has no solution? Using Exercise 8.20, show that the 
overall complexity of using Khachian’s algorithm is O[(m + n) L], where L is 
given by Equation (8.1). 

[8.24] Consider the open inequality system in R? given by x, + Oxy <1+1/0 and 
=x, +0x,<-1, where @>5/V2. Let xg =(1+(V2/5), 0)’ and define 
Eo = {x :(x—Xp)' Qo! (x—x9) <1}, where Qo =2I so that Ep is a circle with 
center at xq and with radius a2: (Hence, this corresponds to “LZ” = 1/2.) Using this 
as a starting solution, apply Khachian’s algorithm (Exercises 8.12—8.17) using A, 
= 0. (This corresponds to Khachian’s original algorithm.) 

a. Give the geometric interpretation of using the formulas for x;,; and 
Qi4; of Exercise 8.14 with 2, = 0. Illustrate with a sketch similar to 
Figure 8.8. 

b. Derive closed form expressions for x, and Q, as a function of 
iteration k. 

c. Determine the smallest integer k such that x, satisfies the inequality 
system. Show that by making @ arbitrarily large, we can make the 
algorithm go through an arbitrarily large number of iterations before 
terminating. What does this say about the complexity of Khachian’s 
algorithm using real data? 

d. Suppose that @ is an integer. Make the coefficients of the open 
inequality system to be integer-valued and determine L from Exercise 


8.12. What does Part (c) say about the practical computational 
relationship of Khachian’s algorithm with the quantity L? 


[8.25] Show that the matrix PP’ is invertible, where P is defined as in Equation (8.7), 
with A being m x n of rank m and with D, =diag{x,1,...,X,,}, given x, >0. 


[8.26] The Karush—Kuhn—Tucker optimality conditions for Problem (8.8) assert 
that the negative gradient of the objective function should be contained in the cone 
spanned by the gradients of the binding constraints. Use this fact to show that 
Ynew defined by Equation (8.9) solves Problem (8.8). 


Complexity of the Simplex Algorithm and Polynomial-Time Algorithms 441 


2 
[8.27] Consider the problem of minimizing le" - Pw over W unrestricted in 


sign. Show by setting the gradient of the objective function equal to zero that 
w =(PP') Pe! solves this problem. 


[8.28] Use Exercise 8.10 to show that there exists a value of M of magnitude gO) 
such that the artificial variable y,,,3; in the problem of Equation (8.13) is zero at 


optimality, given that the problem in Equation (8.12) is feasible. 


[8.29] Determine the value @* of @ that minimizes the expression -@ +a" / 
2(1- a) in Equation (8.22). Why does this not necessarily imply that the value @* 


is the best value for obtaining a maximum reduction in the potential function F(-)? 
How would you go about finding a better value of @ from the viewpoint of 
reducing F(-)? (Hint Examine the derivation of the inequalities in Equations 
(8.20) and (8.21).) 

[8.30] The Karush—-Kuhn—Tucker optimality conditions for Problem (8.24), where 

the constraints are written as < inequalities, require the gradient of the objective 

function to lie in the cone spanned by the gradients of the binding constraints. 

Show that at optimality for Problem (8.24), the first constraint is binding and that y 

> 0. Using the stated optimality conditions, show that at optimality in Problem 

(8.24), some 1 < g < (a — 1) components of y are equal to a value U and the 

remaining components are equal to a value V, where U and V satisfy Equation 

(8.25). 

[8.31] a. Using the procedure of Exercise 8.21, provide the details and the 
complexity analysis of an optimal rounding routine for Karmarkar’s 
algorithm as applied to Problem (8.4) under Assumptions (Al) and 
(A2). In particular, define Z as in Equation (8.1). Show that it is 
sufficient to determine a solution x, to Problem (8.4) with objective 


function value ex, <2 in order to round this solution by using 


Exercise 8.21 to an optimal solution to Problem (8.4) (with a zero 
objective function value) in polynomial time. 

b. Illustrate the method of Part (a) by providing the details of optimally 
rounding the solution x2¢ obtained for Example 8.2. 

c. What is the relationship of this procedure with the rounding routine 
given in Section 8.4? 

d. Present a simplex pivot type of scheme for efficiently implementing 
the optimal rounding routine of Section 8.4. Show that this process 


has a complexity bound of O(n? n). 


[8.32] Consider the potential function f(x) = In[ex/1x]. Show how this trans- 


forms in the y-space under the projective transformation (8.5). Can this potential 
function be used to analyze the complexity of Karmarkar’s algorithm? (Hint 


Examine the relationship between f (x) and ex for Problem (8.4).) 
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{8.33] Analyze in detail the complexity of a single iteration of Karmarkar’s 
algorithm for Problem (8.4). 


[8.34] Using the development of Sections 8.4 and 8.5, show how you can use 
Karmarkar’s algorithm to determine whether a given linear program is infeasible or 
whether it is unbounded. 


[8.35] Provide the details of Karmarkar’s algorithm and its complexity for solving 
Problem (8.4) via Problem (8.27) and the sliding objective method, under 
Assumption (Al) but treating the optimal objective value as unknown. Include an 
analysis of the optimal rounding via the purification step. 


[8.36] Provide the complexity of solving the (general) linear programming 
problem (8.11) via Karmarkar’s algorithm using the transformation of this problem 
into Problem (8.13) and the details of Exercise 8.35. 


[8.37] Illustrate the procedure of Exercise 8.35 using Example 8.4 in Section 8.5 
with the information that the optimal objective value v" €[—6,0]. 


m 


[8.38] Consider the linear assignment problem to minimize pe CyXy subject 


to ey = 1 for j = 1,...,m, vey =1 for i= 1,.... m, and x > 0. Suggest a 


suitable transformation to use Karmarkar’s algorithm for solving this problem. 
Provide an algorithmic statement, including a suggested starting solution and any 
algorithmic specializations. In particular, how would you exploit the fact that the 
extreme points are integer valued? (Hint: With c integer, once a solution within a 
unit of optimality is found, it can be optimally rounded by purification.) 


[8.39] Consider the linear transportation problem to minimize Leta Cy Xij 
subject to sy <s; for i= 1,..., NS, aS xy = d; for j = 1,..., ND, and x > 0, 
where NS is the number of sources with respective supplies s;, i = 1,...,WS, and 


where ND is the number of sinks with respective demands d,;, j = 1,....ND. 
(Assume all integer data and that bay S; 2rd j-) Repeat the analysis of 
Exercise 8.38 for this problem. 

[8.40] Consider the following QR factorization method for obtaining the least— 


squares solution W to the problem of minimizing le ~ iw in order to determine 
c, of Equation (8.9b) via ¢, =¢' —P’w. Given P’ of full column rank, its OR 
decomposition yields P’ = QR, where Q is an n x n orthogonal matrix (that is, 
QQ’ =I) and Ris ann x (m+ 1) matrix of the form ak Here, R, is (m+ 1) x 
(m + 1), nonsingular, and upper triangular. (See the Notes and References section 


F : at = c : 
for details on performing such a decomposition.) Denote Q’¢’ -(¢ | with ¢ 
2 
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being of dimension (m + 1) and ¢c, being of dimension (n — m — 1). Using this 
decomposition, show that w is given as the solution to the linear, triangular 


system R)W = ¢;. Furthermore, show that ¢,, = Q fe | 
2 


[8.41] Suppose that you use the techniques of Sections 8.4 and 8.5 to determine a 
basic solution to the primal—dual formulation in Equation (8.14). Does this 
necessarily yield an optimal primal basis whose complementary dual basis is dual 
feasible? If not, then under what conditions will this be obtained automatically, and 
what additional work may be required otherwise? Give the details of a polynomial— 
time scheme for finding such a basis. Explain the significance of being able to 
obtain such a basis. 


[8.42] Consider the linear programming problem to minimize ex subject to Ax = b 
and x > 0, where A is m x n of rank m, the feasible region is bounded, and where 
the data is all integer. Suppose that the optimal objective value v" and some 
(rational) feasible solution x > 0 to this problem are known. 


a. Show that by using the transformation 
Dx 1 
en eet | a Te 
1D °x+1 1D x+1 
and by replacing the resulting fractional objective by the objective 


function ecDy —v" yp, one obtains an equivalent linear program of the 


form (8.4). 

b. Establish the polynomial-time complexity of Karmarkar’s algorithm 
with respect to the original problem using this approach. 

c. Is the equivalence of Part (a) true if the feasible region of the original 
problem is unbounded? 

d. Consider the following linear programming problem: 


Minimize -x, — 2x 

subjectto x -— X72 + %3 = 1 
x| + 2Xp + x4, = 2? 
x, x2, X%3, X4 2 0. 


Using xX = (0.5, 0.5, 1, 0.5)’ and v" = —2, show that this problem is 
equivalent to that of Example 8.2. 


[8.43] Consider the linear programming problem P given in Equation (8.28), 
where A is m x n of rank m <n and where the data is all integer. Without loss of 
generality, assume that the feasible region is bounded and that a feasible solution 
Xg >0 is known. (Show how one may use artificial variables and bounds, if 
necessary, for this purpose.) Construct the so—called barrier function problem 
defined by Equation (8.35) for a given barrier parameter 2 > 0. Consider the 


following Affine Scaling variant of Karmarkar’s algorithm: 
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Initialization. Let xq be the given positive feasible solution, set k = 0, g = 0.97, 
and select uw = min{2~24~! /n, 107!73, where L = | 1+¥;E log +|ay)]. 

Step 1 (Dual estimates, projected gradient, and direction of motion, based on a 
projected gradient step in the y-space, where y = D;!x. The latter transformation 
gives the algorithm its name.) Given x;, define D, =diag{x;,),....%4,}, ( =eD,, 
and P, = AD,. Compute the following: 


Dual estimate: w, =(P, Pi) 'P, [ef — u1"] 
Reduced cost vector: R, =e! — wD;'1' — A‘w, 
Projected gradient vector: ¢,, =[I— Pg (P,P; y Pe - 1") 
Direction of motion: dz =—Djc px. 
Step 2 (Termination test). If w, is dual feasible in Problem (8.28) and 


ex, —b'w, < 2°*4, then purify x, to an extreme point of Problem (8.28) and 
terminate. Otherwise, continue. 
Step 3 (New iterate.) Set x;,; =x, +4,d,, where the step length A, is given by 


2. 
24 = mining 9) ; 
Lu 


Peas dy 
where = maximum, —-—: j =l,...,7>. 


Xkj 


Amax 


Increment k by one and return to Step 1. 


a. Show that the algorithm generates a sequence of positive iterates x; , 
k=0, 1, 2,.... 
b. Show that the algorithm either stops finitely with an optimal solution 


x; to Problem (8.29) in case le px | =0 for some f, or else, ignoring 
the termination test at Step 2, an infinite sequence of primal—dual 
iterates (x,,W,) is generated such that the sequence {le pk |} > 0. 


Moreover, for any convergent subsequence {x,,w;}x indexed by K, 


with limit point (x,w), show that x > 0 and that x solves Problem 
(8.29). 

c. Show that w obtained in Part (b) is dual feasible to Problem (8.28). 
Moreover, show that the duality gap between x and w in Problem 
(8.28) is ex —b'wW =n. 

d. Show that the reduced cost vector R, approaches zero as k + , and 


that the dual iterate w, becomes dual feasible in Problem (8.28) once 
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R, is sufficiently close to zero. Hence, with uz defined as previously, 
show that the algorithm terminates finitely at Step 2. 
e. Present a purification process for obtaining a pair of optimal primal 
and dual extreme point solutions to Problem (8.28) after termination 
at Step 2. 
f. Comment on the use of 4 = 0. (This was the original affine scaling 

algorithm.) 
[8.44] Consider the linear programming problem given in Exercise 8.42d. Starting 
with the solution xg = (0.5, 0.5, 1, 0.5)’ apply the affine scaling algorithm of 
Exercise 8.43 to solve this problem. Show the progress of the iterates with respect 
to feasibility and objective values in Problems P and D of Equation (8.28), and 
purify the resulting solution to a pair of optimal primal and dual extreme point 
solutions to Problems P and D, respectively. 
[8.45] Suppose that the system Ax = b, x > 0 has a positive feasible solution. 
Consider the following Phase I problem, where xp > 0 is arbitrarily chosen and 
is an artificial variable: 

Pl: Minimize {A : Ax —(2—A)b—(A-I)Axy =0,x 20,/ 2 O}. 
Note that (x,2) =(x9,2) >0 is a feasible, positive solution. Show that P| has an 
optimal value that is strictly less than one. Also, show that for any feasible 
solution (x,2) to Pl, where A <1, the solution x = [(1-2) xo +XJ/(2-A) is a 
positive, feasible solution for the given linear system. Using Exercises 8.42 and 
8.43, suggest a two-phase procedure for Karmarkar’s algorithm, as well as for its 
affine scaling variant. 
[8.46] Show that the dual iterate w, for the affine scaling algorithm of Exercise 
8.43 is the solution to a certain least-squares problem. Provide a geometric inter- 
pretation for this latter problem. 
[8.47] Solve the problem of Exercise 8.2 using the affine scaling algorithm of 
Exercise 8.43, starting with the solution (x1,x2,%3,x4) = (1, 1, 1, 1). Repeat this 
using the affine scaling algorithm (with z= 0) described in Section 8.6. 
[8.48] Consider the linear programming problem: 
Minimize {ex: Ax =b, —1 Sx, Sl for j =1,...,7}, (8.46) 


where A is m x n of rank m and where the data is all integer. Show how a 


problem having general lower and upper bounds @; and w; on the x;- 


variables, where —-o</;<u;<, can be put into this form. Assume (using 


artificial variables if necessary) that there exists some known feasible solution 
Xg to Problem (8.46) satisfying a! < Xo <1‘. Analogous to Exercise 8.43, 


construct the following barrier function problem for Problem (8.46) that uses a 
barrier term to keep the variables strictly within their bounds: 
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n 
Minimize (es -ynY [logdi+ x; )+log(l— x; )]: Ax = »} (8.47) 
j=l 


Here, 4s > 0 is of magnitude 70d): and is assumed to be sufficiently small so 
that a solution near enough to an optimum to Problem (8.47) can be purified to 
an optimal solution to Problem (8.46). Consider the following Bounded 
Variables Affine Scaling Algorithm. 


Initialization. Let -1' < xq <1’ be the initial feasible solution. Set the iteration 
counter k = 0, and select gq €[0.97,0.99] and w= 20): 
Step 1. (Dual estimates, projected gradient, and direction of motion, based on a 
projected gradient step in the y-space, where y = D;'x.) Given x;,, define 
D, = diag{D,,,...,D,,}, where Dy = min{l +Xy 1— x45} for 7 = 1,...,., and 
define d(x; ) =[6) (x4 1),--6), (Xin I where 6 ; (x4) = (-2D yxy) xf) for j = 
1,..., 2. (Note that -1< 6 (x4) <1 forj = 1,..., 2.) Denote ¢, =cD,, P, = AD,, 
and compute the following: 

Dual estimate: w;, = (P,P; ye P(t — ud(X,)] 

Reduced cost vector: R, =e! — wD;'5(x;)- A'w* 

Projected gradient vector: ¢,, = D,R, =[I- Pi (P;, Py ee P, IG, — 6(x;)] 

Direction of motion: dy =—D,¢,, . 
Step 2. (Termination test.) If le pk | =0, then stop with x, as an optimal solution 
to Problem (8.47); purify x, to an optimal vertex of Problem (8.46). (Practically, 
one would stop if le pk | is sufficiently small.) 
Step 3. (New iterate.) Set x,,) =x, +4,d, where the step length A, is given by 

Ay = minimum {aAmax(l eae /4u} 


where 


1 d,; di; 
= maximum] max ie AS is 4 >0 
Amax J=1,.... (I — XK) (i+ xy) 
Increment k by one and return to Step 1. 


a. Show that the bounded variables affine scaling algorithm either stops 
finitely with an optimal solution x, to Problem (8.47) in case 


le pk |=9. or else, an infinite sequence of primal and dual iterates 


{x,,W,} is generated such that the sequence {le pk |} — 0. Moreover, 
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for any convergent subsequence {x,;,wW,}x indexed by K with limit 


point (x, W), we have -1' <x <1‘ and that X solves Problem (8.47). 
b. Show that the dual iterates w, correspond to a dual feasible solution 
to Problem (8.46) for all &. Obtain an expression for the duality gap 
between the primal and dual solutions x, and w,; for Problem (8.46). 
Suggest a termination criterion based on this duality gap that will 
permit a purification to a pair of optimal primal and dual extreme 
point solutions for Problem (8.46). 
[8.49] Solve the following problem using the bounded variables affine scaling 
algorithm, after transforming it to the form required by Exercise 8.48: 
Minimize — x 
subjectto 2x, + XxX. + %3 
Xx -— xX + X% 


2 
4 


-lsx, <2, -3sx,s-l, 0S5%,<57, OSx4 83. 
Use the solution x = (0, -2, 4, 2)' as the starting solution. 


[8.50] For the bounded variables affine scaling algorithm of Exercise 8.48, 
consider the following alternative affine transformation to be used in Step 1 of the 
algorithm: 


x=D,y where D,; = diag |(1- xg \; J= rene 
Using the same barrier function problem (8.47), state an affine scaling algorithm 
analogous to the one in Exercise 8.48 and establish a similar convergence result. 
[8.51] Illustrate the algorithm of Exercise 8.50 using the example of Exercise 8.49. 
[8.52] Consider the problem of estimating the parameters of a linear regression 
model b = Ax — E, where b = (4),...,5, y is a vector of n observations, A is an x 


m matrix with each row corresponding to values of some m independent variables, 
x is a vector of m parameters to be estimated, and E is a vector of n random errors 
that are independently and identically distributed according to some distribution. 
We are interested in finding out a set of values for the parameters x that minimize 


n 
rill: 
a. Formulate this problem as a linear programming problem. 
b. Write the dual to this problem and show how the bounded variables 
affine scaling algorithm of Exercise 8.48 can be used to solve this 
problem. 


[8.53] Consider the system in Equation (8.41). Denoting X, and V;, as diagonal 
matrices having the respective components of x, and v, as the diagonal ele- 
ments, verify that Equation (8.41c) (without the final quadratic term) can be 
equivalently rewritten as follows: 


Ved + Xpdy = gy l —VX AV’, 
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where 1’ is a column vector of ones. Hence, show that the solution to this along 
with Equations (8.41a and b) yields d gé uniquely via: 


di, = (AV, XA‘) AVE EV, Xl = Apt] 
di =-A'di, 
d= Vg! [aga -V,X_l' -— X; 04]. 
[8.54] In the primal—dual path—following algorithm, suppose that & = (x;, W;, 
v,) satisfies Equation (8.39), where = 0.35. Revising “4, to 4,4, according to 
Equation (8.40), where / is given by Equation (8.43), let d g be given via the sys- 
tem in Equation (8.41) (see Exercise 8.53), and define &,,) as in Equation (8.42). 
Show that &,,, and 4,4; satisfy the conditions (8.39) with & replaced by k+ 1. 
[8.55] Analyze in detail the complexity of a single iteration of the primal—dual 
path-following algorithm, and show that it is polynomially bounded by O(n). 


(This can be reduced to a complexity order of O(n?) using an argument due to 


Karmarkar of updating the solution to the system in Equation (8.41) from one 
iteration to the next, in lieu of solving it from scratch at each iteration.) 


[8.56] Consider the problem to minimize 2x, +3xz +3, subject to 3x, + 2x. + 
4x, =9, x > 0. Solve this problem using the primal—dual path—following method 
using a starting solution (A = 0) of x, =(1, 1, 1)’, w, =—l1, where w is the dual 
variable associated with the single equality constraint, and 4, = 5. Is this starting 
solution on the central path? 


{8.57] Resolve the example of Exercise 8.56 using the predictor—corrector path— 
following algorithm described in Section 8.6, starting with the same solution as 
given in Exercise 8.56 and adopting the following rule to update the parameter su: 


_[vie¥ee 
AV; Xx ; 


Hk+1 


Give an interpretation of this formula. (Hint: Observe that 


t t 2 
_ | Ve+i Xho VEXk 
Hx+y =| ———— | | I] 
VX n 


NOTES AND REFERENCES 


1. The concept of “good” or polynomially bounded algorithms was 
independently proposed by Edmonds [1965], and Cobham [1965]. See 
Cook [1971], Karp [1972], Garey and Johnson [1979], and Papadimitriou 
and Steiglitz [1982] for further reading on this subject. 
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2. For the complexity analysis of the simplex method under various pivoting 
rules, see Klee and Minty [1972], Jeroslow [1973], Avis and Chvatal 
[1978], and Goldfarb and Sit [1979]. (Exercise 8.8 uses Chvatal’s [1983] 
approach.) 

3, Around 1957, Warren Hirsch conjectured (see Dantzig [1963a]) that for 
bounded polyhedral sets with f facets in n dimensions, any two vertices of 
the polytope are connected by a simplex path of length at most (f- 2). 
This is referred to as the Hirsch conjecture. Klee and Walkup [1967] 
showed the necessity of boundedness of the polyhedral set and established 
this conjecture under the restriction f— n < 5. (Klee [1965a] showed this 
conjecture to be true for » < 3.) The monotonic—bounded Hirsch 
conjecture asserts that there exists a simplex path between any two 
vertices of length no more than (f — x), which additionally has 
nonincreasing objective function values. Todd [1980] has shown this 
conjecture to be false for 7 > 4. For a related result on the zero—one set 
partitioning problem, see Balas and Padberg [1970]. 

4. Average—case probabilistic complexity analyses of the simplex method 
appear in the novel work of Borgwardt [1982a, b] and Smale [1983a, b]. 
This independent work has been coordinated and extended by Haimovich 
[1983]. Also, see Todd [1989] for a survey. 

5. The Soviet mathematician L. G. Khachian [1979] was the first to intro- 
duce linear programming problems in the Class P of problems. Proofs of 
the results appear in Khachian [1981] and Gacs and Lovasz [1981]. The 
latter paper contains the ideas of Exercises 8.14, and the optimal rounding 
scheme of Exercise 8.21. Khachian’s ellipsoid algorithm derives its ideas 
from earlier work by other Soviet mathematicians including Shor [1970a, 
b]. Analysis of this algorithm using finite precision arithmetic appears in 
Khachian [1980] and Grotschel et al. [1981]. The “deep cut” variation 
discussed in Exercises 8.12-8.17 was first presented in Shor and 
Gershovich [1979]. The unbounded complexity of Khachian’s algorithm 
with real, noninteger data was demonstrated by Traub and Wozniakowski 
[1982] (see Exercise 8.24). A polynomial variant of Khachian’s algorithm 
that uses simplices in lieu of ellipsoids appears in Yaminitsky and Levin 
[1982]. An excellent survey appears in Bland et al. [1981]. Also, see the 
survey paper of Shor [1983] and the development of the ellipsoid 
algorithm in Murty [1983] and Schrijver [1986] for further details. 

6. Using a polynomial-time linear programming subroutine, Tardos [1985, 
1986] showed that the linear program to minimize cx subject to Ax < b 
with integer data can be solved in a number of elementary arithmetic 
operations bounded by a polynomial in the size of A alone. Hence, 
network structured problems, for example, admit genuinely or strongly 
polynomial algorithms. Megiddo [1983] also derived a strongly polyno- 
mial algorithm for linear programs having at most two nonzero 
coefficients in the objective function and in each of the constraints. 

7. The projective polynomial-time algorithm for linear programming was 
proposed by N. Karmarkar [1984a, b, 1985] at AT&T Bell Laboratories, 
U.S.A. Lawler [1985] and Padberg [1986] discuss the use of larger step 
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sizes, and Goldfarb and Mehrotra [1988] and Anstreicher [1986a] validate 
techniques using approximate projection operations. An alternative to 
using the sliding objective function/bisection scheme is presented in Todd 
and Burrell [1986] and Anstreicher [1985]. Charnes et al. [1984] exhibited 
a worst-case linear convergence rate for Karmarkar’s algorithm. However, 
Iri and Imai [1986] developed a modification that is quadratically 
convergent. For additional analyses on superlinearly convergent primal— 
dual path-following methods, see Mehrotra [1993], Tapia et al. [1995], Ye 
et al. [1993], Zhang et al. [1992], and Zhang and Tapia [1993]. 
Implementation suggestions and transformations for solving general linear 
programs by Karmarkar’s method appear in Tomlin [1985] and Shanno 
and Marsten [1985], in particular. The two-phase approach of Exercise 
8.45 is from Kojima [1986]. For factorizations in solving the least-squares 
problem accurately, see Dongarra et al. [1979], Heath [1982], George and 
Heath [1980], and Todd and Burrell [1986]. Relationships of Karmarkar’s 
algorithm to projected Newton barrier methods have been shown by Gill 
et al. [1986], and to the ellipsoid algorithm have been demonstrated by 
Todd [1989] and Ye [1987]. For computational experience using linear 
assignment problems, see Aronson et al. [1985]. The purification scheme 
first found its elements in Charnes et al. [1965]. Also, see Kortanek and 
Jishan [1988]. Sherali et al. [1988] showed how to use such a scheme for 
optimally rounding to a basic feasible solution. Ye [1987] also discusses 
how to obtain an optimal basis via Karmarkar's algorithm, and Ye and 
Kojima [1987] discuss how to recover dual solutions. Variants and 
discussions related to Karmarkar’s projective algorithm appear in 
Anstreicher [1986b, c], Asic et al. [1986], Blair [1986], Blum [1985], 
Chang and Murty [1987], de Ghellinck and Vial [1986], Gay [1987], 
Gonzaga [1985, 1987b], Kojima [1986], Lustig [1985], Megiddo [1985, 
1986a, b], Murty [1985], Nazareth [1986], Rinaldi [1986], Ye [1985a, b], 
and Chiu and Ye [1985b], among others. (See Note 8 for the affine scaling 
and the path-following variants.) For exploiting special structures using a 
variant of Karmarkar’s projective algorithm, see Todd [1988]. Anstreicher 
[1997], Freund and Mizuno [1996], and Monma [1987], provide survey 
articles; see also Martin [1999], Saigal [1995], Terlaky [1998], and 
Vanderbei [1996]. 

The affine scaling variant of Karmarkar’s algorithm finds its origin in 
Dikin [1967, 1974], but was rediscovered independently by Barnes 
[1986], Vanderbei et al. [1986] and Cavalier and Soyster [1985], follow- 
ing Karmarkar’s algorithm. Barnes [1986], Vanderbei et al. [1986], 
Kortanek and Shi [1987], and Sherali [1987] prove convergence under 
various conditions. Sherali et al. [1988] have presented an assumption— 
free convergence analysis of a perturbation of this algorithm. (Exercises 
8.43, 8.48, and 8.50 are derived from this paper.) For exploiting special 
structures using this variant, see Chandru and Kochar [1986]. Adler et al. 
[1986] and Monma and Morton [1987] have also independently developed 
dual affine variants of Karmarkar’s algorithm. Megiddo and Shub [1987] 
have exhibited a worst-case exponential behavior for the affine scaling 
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algorithm with infinitesimal steps (continuous trajectory) on the Klee— 
Minty [1972] polytope by starting the algorithm close enough to a vertex. 
See Saigal [1995] and Vanderbei [1996] for detailed analyses regarding 
the convergence of the affine scaling method under degeneracy. By 
adopting an additional step that centers the iterate within the polytope, 
Barnes [1987] has derived a polynomial-time variant of the affine scaling 
algorithm. This centering amounts to increasing the denominator of 
Karmarkar’s multiplicative potential function, while the usual gradient— 
based step is aimed at reducing the numerator of this potential function. 
Hence, we obtain a polynomial-time behavior. This approach falls under 
the category of path—following algorithms. These algorithms have been 


motivated by the work of Megiddo [1986b], and include the O(n°L) 


algorithms of Ben Daya and Shetty [1988], Gonzaga [1987a], Monteiro 
and Adler [1989a, b], Renegar [1988], Vaidya [1987], and Ye and Todd 
(1987], among others (see Terlaky [1998] and Martin [1999] for further 
references and discussions). In particular, McShane et al. [1988] provide 
implementation details and computations using the primal—dual path— 
following method of Monteiro and Adler [1989a], which is discussed in 
Section 8.6. The work of Gonzaga [1987a, b] also motivates a 
bidirectional search in the y-space based on the projection of the vector 1, 
in addition to that of the vector —cD,, onto the null space of P, = AD, in 


the context of the affine scaling method. Gonzaga [1988] also shows how 
to derive a polynomial-time affine scaling variant through a potential 
function minimization process. For a polynomial-time primal—dual affine 
scaling variant of Karmarkar’s algorithm, see Monteiro et al. [1990]. The 
popular predictor—corrector variants of primal—dual path—following 
algorithms were introduced by Mehrotra [1991, 1992], and 
computationally implemented by Lustig et al. [1992a, b]. Kojima et al. 
[1993] (see implementation aspects in Lustig et al. [1994a, b]), and Zhang 
and Zhang [1995] provide convergence analyses and polynomial 
complexity proofs for such variants. Carpenter et al. [1993] explore 
higher-order variants of predictor—corrector methods. For extensions of 
interior point methods to quadratic and convex nonlinear programs, see 
the exposition given in den Hertog [1994], Nesterov and Nemirovskii 
[1993], and Yudin and Nemirovskii [1976]. 

9. Similar to Khachian’s [1979] algorithm, Dunagan and Vempala [2008] 
discuss an algorithm to solve linear programs by finding a solution to a 


system of linear inequalities. Given m linear inequalities in R”, assumed 
to contain a ball of radius 9, they devise a randomized perceptron—like 


algorithm (from machine learning) based on a periodic re-scaling 
technique to find a feasible solution with probability (1 — 6), where 0 < 6< 


1, in polynomial time with complexity O(mn* log(n) log(/p9 ) + 
mn’ log(n) log(1/6)). 
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NINE: MINIMAL COST NETWORK 
FLOWS 


In this chapter, we present a specialization of the simplex algorithm to network 
structured linear programming problems. This specialization, known as the net- 
work simplex algorithm, performs the simplex operations directly on the graph or 
the network itself. We also discuss appropriate data structures that facilitate the 
implementation of such a graph-theoretic procedure on the computer. The overall 
efficiency with which such a procedure operates enables one to solve problems 
200-300 times faster than with a standard simplex approach that ignores any 
inherent special structures other than sparsity. Hence, very large problems can be 
solved with a reasonable amount of effort. Note that the network simplex 
algorithm has been shown to exhibit worst-case exponential behavior on certain 
classes of transportation problems, although new polynomial-time variants are 
emerging. Moreover, there exist (strongly) polynomial-time algorithms for 
network flow problems based on an iterative scaling method or on an 
implementation of a dual simplex method, as well as specialized types of 
polynomial-time primal simplex algorithms. Also, Khachian’s and Karmarkar’s 
polynomial-time algorithms could be used, specialized to exploit the coefficient 
matrix form and sparsity. However, the algorithms discussed in this chapter and in 
Chapter 1] are by far the most effective in practice. Furthermore, this chapter lays 
the foundation for studying polynomial-time dual—simplex algorithms, as well as 
more advanced topics dealing with generalized networks, extracting network or 
generalized network structures or substructures from given linear programming 
problems, and solving network flow problems with side—constraints and/or side— 
variables. We refer the reader to the Notes and References section for reading 
material on these topics. 

We begin our discussion by introducing network structured linear pro- 
grams and some associated concepts from graph theory that are relevant to our 
study. Next, we present an important property possessed by the coefficient 
matrix that gives this problem its special structure, and we characterize a basis 
matrix in graph—theoretic terms. This leads us into developing the network sim- 
plex algorithm for the nonnegatively constrained as well as for the bounded 
variables problem. Thereafter, we examine data structures for representing the 
basis matrix that enable us to implement simplex types of algorithms on the 
computer. This discussion includes terminology and concepts that are important 
in educating the reader sufficiently to study much of the modern literature on 
network flow programming. 


9.1 THE MINIMAL COST NETWORK FLOW PROBLEM 


Consider a directed network or a digraph G, consisting of a finite set of nodes 
(vertices or points). / = {1, 2,..., m} and a set of directed arcs (links, branches, 


edges, or lines) . >= {(i, /), (k, 4),...,(s, D} joining pairs of nodes in. ”. Arc (i, 
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J) is said to be incident at nodes i and j and is directed from node i to node j. We 
shall assume that the network has m nodes and » arcs. Figure 9.1 presents a 
network having four nodes and seven arcs. 

With each node i in G we associate a number 5; that represents the 
available supply of an item (if b; > 0) or the required demand for the item (if 5; < 
0). Nodes with 5; > 0 are called sources, and nodes with 5; < 0 are called sinks. 
If b, = 0, then none of the item is available at node i and none is required; in this 
case, node i is called a transshipment (or intermediate) node. Associated with 
each arc (i, 7) we let x; be the amount of flow on the arc (we assume xy 2 0), 
and we let cj be the unit shipping cost along this arc. 


We shall assume that the total supply equals the total demand within the 
network, that is, 27,5; =0. If this is not the case, that is, "5; > 0, then we 


can add a dummy demand node, m + 1, with 5,,,) = oo we 15; and arcs with zero 


cost from each supply node to the new node. 

The minimal—cost network flow problem may be stated as follows: ship 
the available supply through the network to satisfy demand at minimal cost. 
Mathematically, this problem becomes the following (where summations are 
taken over existing arcs): 


mmm 
Minimize > > CyXjj 
i=l j=l 
m m 
subjectto > Xi xy =5, i=l,..,m (9.1) 
j=l k=l 


x, 


ij 20, ij=l..,m. 


Constraints (9.1) are called the flow conservation, or nodal balance, or Kirchhoff 
equations and indicate that the flow may be neither created nor destroyed in the 


network. In the conservation equations, Liat represents the total flow out of 


node i, while ))7",x,; indicates the total flow into node i. These equations 
m 
J= 
particular, if b; < 0, then there should be more flow into i than out of i. This 


problem is also said to be uncapacitated. The problem that includes upper 
bounds on arcs is said to be capacitated and is considered subsequently. 

The minimal cost flow problem might arise in a logistics network in 
which people and materials are being moved between various points in the 
world. It may be associated with the movement of locomotives between points 
in a railroad network to satisfy power for trains while minimizing travel costs. 
Minimal cost network flow problems occur in the design and analysis of 
communication systems, supply chain and distribution problems, oil pipeline 
systems, tanker scheduling problems, and a variety of other areas. 


require that the net flow out of node i, 077) xj — Ly_1X4;, should equal 4;. So, in 
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Figure 9.1. Example of a network. 


Clearly, the minimal cost flow problem is a linear program and can be 
solved in any one of several ways. One way is to apply the ordinary primal 
simplex algorithm to the problem. What we seek in this chapter is a simplifica- 
tion of the simplex method so that it can be applied directly on the network 
without the need of a simplex tableau. 


9.2 SOME BASIC DEFINITIONS AND TERMINOLOGY FROM 
GRAPH THEORY 


Consider the digraph G(. / ',.°7) introduced earlier with node set . / “and arc 
set . « This graph is said to be proper if the cardinalities of . / ‘and . >satisfy 
|. / |22 and|.:/|2>1. Two nodes in the graph are said to be adjacent if they 


are directly connected by some arc. For the directed arc (i, j)€.°/%, node i is 


called its from—node and node ; is called its to—node. For any node i, the set of 
nodes j for which there exists an arc (i, /) is called the forward-star of i. 
Similarly, for any node i, the set of nodes 7 for which there exists an arc (j, i) is 
called the reverse-star of i. 

Throughout our study, we will be concerned with graphs that have 
directed arcs only, that is, with digraphs. In contrast, an arc that has no 
orientation and that permits flow in either direction is called an undirected arc. 
A graph that has all undirected arcs is called an undirected graph. If a graph has 
both directed and undirected arcs, then it is called a mixed graph. Note that in 
the context of the minimum cost network flow problem, if there exists an 
undirected arc (i, 7) with a cost cj 2 0, then this can be equivalently replaced by 


two oppositely oriented directed arcs (i, /) and (/, i), each with the same unit cost 
cj. However, this construction is not valid if cy < 0 for flow in either direction. 


In such a case, if the problem is feasible, then we can take any feasible solution 
to the original problem and superimpose on it an infinite circulation of flow 
from i to j along (i, /) and back from / to i along (j, i‘) in the transformed 
network. In this manner, we can maintain feasibility in Problem (9.1) (assuming 
that this is feasible) and drive the objective function to —co, while in reality, such 
a circulation of flow results in a zero additional flow and cost in the original 
problem. 

Continuing, a path from node ig to i, is a sequence of arcs P = {(ig,4)), 


(4, i2),---s(ip1s#p)} in which the initial node of each arc is the same as the 
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terminal node of the preceding arc in the sequence and jg,...,i, are all distinct 
nodes. Thus each arc in the path is directed “toward” i,, and “away from” ip. A 


chain is a similar structure to a path except that not all arcs are necessarily 
directed toward node i,,. Figure 9.2a illustrates a path, and Figure 9.2b presents 


a chain. A circuit is a path from some node ig to ip plus the arc (i,i9). Thus, a 


circuit 1s a closed path. Similarly, a cycle is a closed chain. Figures 9.2c and 
9.2d depict circuits and cycles. Every path is a chain but not vice versa. Every 
circuit is a cycle but not conversely. 

These definitions refer to what are known as simple paths, chains, circuits, 
or cycles. We will assume that these definitions apply unless otherwise 
specified. A nonsimple path, for example, can permit nodes in the set ig, i,...,4p 


to repeat and hence, may include circuits. For example, the set of arcs {(1, 2),(2, 
3),(3, 4),(4, 2),(2, 3),(3, 5),(5, 6),(6, 7),(7, 5),(5, 8)} describes a nonsimple path 
from node | to 8, with the sequence of nodes ip, ij,...,7, Visited being ig = 1, 


2,3, 4, 2, 3, 5, 6, 7, 5, and 8 = i,. 


Throughout this chapter, we shall assume that the underlying graph G for 
the minimum cost network flow problem is connected, that is, there exists a 
chain between every pair of nodes in G. This property is also referred to as 
yielding a weakly connected graph. A strongly connected graph is one where 
there is a (directed) path from each node to every other node. A complete graph 
is one where each node is connected by an arc to every other node. Hence, a 
complete digraph on m nodes has m(m — 1) arcs. A subgraph G'(. / ",.°/') ofa 


Pp 


graph G(. / ,.°/) is one that satisfies. / “'c. / and.*/’c.’/, with the under- 
standing that if (7, /)¢.°/, then both 7 andj are in . / ’. If G'#G, then G' is 
said to be a proper subgraph of G. If ./ "=. / , then G' is said to be a 


spanning subgraph of G, that is, it “spans” all the nodes of G. A subgraph 
G'C 14.7%") of GO / 4.77) is said to be induced by the node set .{ " if .°/' 


ft 


includes all the arcs in . *that have both their associated nodes present in . / 
For example, referring to Figure 9.1, the subgraph induced by the node set . / ” 
= {1, 2, 4} is the graph having nodes 1, 2, and 4, and with the arc set .v’ = {(1, 
2), (2, 4), (4, 1)}. A component of a graph G is a subgraph that is connected and 
that is not a proper subgraph of another connected subgraph. Hence, the 
components of a graph are maximal connected subgraphs—they are “separable 
pieces” of a graph. For example, if we delete arcs (1, 2), (1, 3), and (4, 1) in 
Figure 9.1, the resulting graph has two components (which ones?). 

A tree is a connected graph having no cycles. We call a tree an 
arborescence if there exists some node r (called a root node) such that there 
exists a (directed) path connecting node r to every other node in the tree. A 
spanning tree, defined with respect to some underlying graph G, is a tree that 
includes every node of the graph, that is, it is a spanning, connected subgraph 
having no cycles. Figure 9.2e illustrates a spanning tree of the graph in Figure 
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(a) A path (b) A chain 
(c) A circuit (d) A cycle 


(e) A tree 


Figure 9.2. Paths, chains, circuits, cycles, and trees. 


9.1. Figures 9.2a and 9.2b are also examples of spanning trees. Figures 9.2c and 
9.2d are not trees. Note that a graph having no cycles has a tree as each of its 
components (why?). Such a graph is called a forest. Similarly, a spanning forest 
defined with respect to a graph G is a spanning subgraph of G having no cycles. 
The rank or degree of a node is the number of arcs incident at it. The in—degree 
of a node 7 is the number of arcs that have i as their to—node, and the out—degree 
of i is the number of arcs that have i as their from—node. Hence, (in-degree of 1) 
+ (out-degree of 7?) = (degree of 7). For example, in Figure 9.1, the in-degree of 
node | is | and its out—degree is 2; its degree is 3. A node having degree 0 or 1 
is an end node. (Sometimes, the end node of a tree is referred to as a leaf node.) 
For example, nodes 1 and 4 are end (leaf) nodes of the tree in Figure 9.2a. 

We shall see shortly that trees play a very central role in network flow 
problems because they help us characterize bases of the coefficient matrix in 
Problem (9.1). Let us examine some additional facts concerning tree graphs. 


Property 1 


Let T be a proper tree graph having m(> 2) nodes, and let (i, /) € 7. Then 
disconnecting (i, j) from T, that is, removing the arc (i, /) from T but leaving the 
nodes i and j in 7, decomposes T into two trees 7, and 7>. This follows since 
disconnecting (i, 7) must result in r > 2 components, each component being a 
tree, or else, by putting back (i, /), we would have a cycle in 7, a contradiction. 
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Because putting back (i, 7) can connect no more than two components by 
definition, and because T is connected, we must have r = 2. 


Property 2 


A proper tree graph has at least two end nodes. This is clearly true for a tree 
having only two nodes. By induction, suppose that this is true for a tree having 
2,...,(m — 1) nodes, and let us show that it is true for a tree having m nodes, 
where m > 3. Select some (i, 7) € T and disconnect it from 7, creating two trees 
7, and 7,, each having no more than (m — 1) nodes. If 7 or 7, has only one 


node, then by the induction hypothesis, the total number of end nodes in 7, and 
T is at least 3 (why?). Moreover, by putting back (i, /), we can lose at most one 
of these end nodes. Hence, T has at least two end nodes. Otherwise, if both 7, 
and 7> have at least two nodes, then they have at least four end nodes in total, 


and putting back (i, 7) can result in a loss of at most two end nodes. This again 
shows that T has at least two end nodes. 


Property 3 


A tree having m nodes has (m — 1) arcs. This is clearly true for m = 1 or 2. By 
induction, assume that this property holds for a tree having (m — 1) nodes and 
consider a tree with m nodes, m > 3. By Property 2, an end node / exists. 
Disconnect the (unique) arc incident at the end node and obtain two trees 7, and 
T, (by Property 1), where 7, = {i}. Hence, 7, has zero arcs, and 7, has (m — 
1) nodes. By the induction hypothesis, it has (m — 2) arcs. Therefore, T has (m — 
2)+1=(m-—1) arcs. 

These fundamental properties help establish several equivalent characteri- 
zations of a tree graph. 


Equivalent Characterizations of a Tree Graph T 


(a) Tis connected and has no cycles. 

(b) Tis connected and has (m — 1) arcs. 

(c) T has (m-— 1) arcs and has no cycles. 

(d) T is connected, but disconnecting any arc from T results in two 
components. 

(e) T has no cycles, but adding any new arc to 7 results in a graph 
having exactly one cycle. (Such a graph is called a one-tree.) 

(f) T has a unique chain connecting each pair of its nodes. 


Observe that Statements (a), (b), and (c) assert that any two of the three 
characteristics of being connected, having no cycles, and having (m — 1) arcs 
implies the third. Furthermore, we shall see later that Statements (d) and (f) play 
a role in computing and updating dual and primal variables. Statement (e) 
pertains to the representation of a nonbasic variable column in terms of the basic 
variable columns. 
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The equivalence of these characterizations is readily established. For 
example, consider Statements (a) and (b). Note that Statement (a) implies State- 
ment (b) by Property 3. Statement (b) also implies Statement (a), because if T is 
connected, has (m — 1) arcs, and contains a cycle, then we can delete an arc from 
the cycle and not destroy connectedness (why?). Using this method, we can 
obtain a connected graph having no cycles, but one that has fewer than (m — 1) 
arcs—a contradiction to Property 3. Hence, Statements (a) and (b) are equiva- 
lent. Similarly, Statements (a) and (c) are equivalent since Statement (a) implies 
Statement (c) by Property 3, and Statement (c) implies Statement (a), because if 
it did not, then we would have a graph T having (m — 1) arcs and with no cycles 
that is not connected. Then, by adding an arc between any two components, we 
can reduce the number of components by one without creating any cycle 
(why?). Continuing in this fashion, we can obtain a connected graph having no 
cycles, but one that has more than (m — 1) arcs—a contradiction to Property 3. 
Hence, Statements (a) and (c) are equivalent. The equivalence of Statement (a) 
with the other characterizations is easily shown in a similar manner using these 
properties. This is left to the reader in Exercise 9.9. 


9.3 PROPERTIES OF THE A MATRIX 


Consider the coefficient matrix A associated with the constraint set (9.1). The 
matrix A has one row for each node of the network and one column for each arc. 
Each column of A contains exactly two nonzero coefficients: a “+1” and a “—1.” 
The column associated with arc (i, /) contains a “+1” in row i, a “—1” in row /, 
and zeros elsewhere. Thus, the columns of A are given by 

ay =e; - ej 
where e; and e; are unit vectors in R™, with ones in the ith and jth positions, 


respectively. The A matrix is called the node-arc incidence matrix for the 
graph. The A matrix for the network of Figure 9.1 is: 


(1,2) (1,3) (2,3) (2,4) (3,2) = (3,4) (4,1) 


if 1 1 0 0 0 0 -1 

ere ee 0 I 1 =I 0 0 
3} 0 = -1 0 I 1 0} 
4| 0 0 0 ~I 0 =I 1 


Rank of the A Matrix 


Assume that A is a node~arc incidence matrix of a connected digraph. Clearly, 
the A matrix does not have full rank, since the sum of its rows is the zero vector. 
To show that A has rank m — 1 we need only select an (m — 1) x (m — 1) 
submatrix from A that is nonsingular. 

Let T be any spanning tree in the network G. By deleting arcs involved in 
a cycle, we can show constructively that such a tree exists. From the earlier 
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Figure 9.3. Reduction of T to T. 


discussion, 7 consists of the m nodes of G together with (7m — 1) arcs of G that 
do not form a cycle. Consider the m x (m — 1) submatrix Az of A associated 


with the nodes and arcs in T. Since m > 2, T has at least one end k. Accordingly, 
the Ath row of A; contains a single nonzero entry. Permute the rows and 


columns of A; so that this nonzero entry is in the first row and first column. 
Then, Ay becomes 


Delete the first row and column of Az and consider the matrix A;’, which is 
(m — 1) x (m — 2). Correspondingly, obtain the graph T’ from T by removing 
node & and the incident arc (see Figure 9.3). Note that 7’ is also a tree. It must 
contain at least one end, say, node £. Permuting the rows and columns of A; so 


that the single nonzero entry in row @ is in the first row and column, we may 
write A; as 


+1 0 
Ar =| Pi 0 
P2 Ar" 


We can continue in this manner exactly m — 1 times, after which all m — 1 
columns of Az are fixed. Deleting the remaining bottom row of Az, we will 


have an (m — 1) x (m — 1) matrix By that is lower triangular with nonzero 


diagonal elements and therefore, nonsingular. Thus, the rank of A is m— 1. 

If we select columns (1, 3), (2, 3), and (3, 4) from the node-arc incidence 
matrix for the network of Figure 9.1, we get the following lower triangular 
matrix after using the foregoing permutation process, with (end) nodes selected 
in the order 1, 2, and 3, and by finally discarding row 4: 


Minimal Cost Network Flows 461 


d,3) 32,3) = 3,4) 


1 0 0 
B, =2| 0 1 0 |. 


3| -1 ~] 1 


The corresponding spanning tree is given as follows: 


The Artificial Variable 


Recall that the simplex method always starts with a full rank constraint matrix. 
We demonstrated earlier that the rank of A is m — 1. Therefore, an artificial 
variable could be added so that the rank of the new matrix is m. Introducing an 
artificial variable corresponding to node m (any other node would do) leads to 
the constraint matrix (A,e,,). We need not penalize the artificial variable, since 


it must be zero in any feasible solution to the artificial problem (why?). 

Any basic solution must contain m linearly independent columns, and 
hence the artificial variable must appear in every basic solution. If we liberalize 
our definition of an arc, then the new column can be viewed as an arc beginning 
at node m and terminating in space (see Figure 9.4). This one—-ended arc is 
called a root arc. The associated node (m) is called a root node. 


Characterization of a Basis Matrix 


We determined the rank of A by examining the submatrix associated with any 
spanning tree. This also demonstrates that a spanning tree together with a single 
artificial variable corresponds to a basis for the A matrix. A graph of this type is 
called a rooted spanning tree and is illustrated in Figure 9.5. Note that the 
associated basis matrix B obtained by adding the root arc and the root node as 
the last column and row, respectively, to By is lower triangular, because By is 


ac Le 


Figure 9.4. A generalized graph G. 
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Figure 9.5. A basis subgraph is a rooted spanning tree. 


lower triangular and the root arc column is e,,. This is shown below for the 
previous example. 


d,3) = =(2,3) ~— (3,4): root are 


a 0 0 0 
2) 0 1 0 0 
ce ee a 1 0 
4) 0 0 = 


A rooted spanning tree corresponds to a (lower triangular) basis for (A,e,, ). The 
converse of this is also true. Namely, any basis for (A,e,,) is a node—arc 


incidence matrix of a rooted spanning tree. To see this, let B be an m x m basis 
matrix of (A,e,,), and let Gp be the graph whose node-arc incidence matrix is 


B. Since the artificial column must be a part of every basis and since B is m x m, 
Gg is a rooted spanning subgraph of G, that is, it is a spanning subgraph plus 
the root arc. Besides the root arc, it has (m — 1) arcs. Furthermore, it must be 
connected or else, it contains a component having no root arc. However, the 
rows of B corresponding to the nodes in such a component must add to zero 
since each basic arc column having nonzero entries in these rows has a + | and a 
—| in these rows and zeros elsewhere. Hence, Gg has m nodes and one root arc, 
it is connected, and has (m — 1) arcs. By the equivalent characterization (b) of a 
tree in the foregoing section, Gg is a rooted spanning tree. 


Although the previous argument implies that Gg cannot contain a cycle, 
it is instructive to see this property more directly. Suppose that Gp is a rooted 
spanning subgraph of G, and by contradiction, suppose that Gg contains a cycle 


(see Figure 9.6). Select some arc (i, ) in the cycle and orient the cycle in the 
direction of that arc. Then, for each column of B associated with an arc in the 
cycle, assign a coefficient of +1 if the arc is in the direction of orientation of the 
cycle and —1 otherwise. Applying these coefficients to the respective columns in 
B, we find that the weighted sum of these columns is the zero vector. In Figure 
9.6 we have 


Ay — Ayj —Apy + Ag +A png +-°- = (C; —e;)— (ex —e;)-(e —e;,) 


+(e, -e€p) + (Cp —€g) +: = 0. 
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Orientation 


Figure 9.6. Illustration of a linearly dependent set of columns. 


Thus, the columns of B could not have been linearly independent. This 
contradiction shows that Gg contains no cycles. But Gg contains m nodes and 


m-— 1 arcs. Hence, Gp is a tree (why?). 


To summarize, we have shown that each basis consists of the root 
together with a spanning tree (see Figure 9.5), and conversely. Thus, we have 
the following theorem holding true. 


Theorem 9.1 


Consider a minimum-cost network flow problem defined on a connected 
digraph G having one root arc. Then B is a basis matrix for this problem if and 
only if it is the node—arc incidence matrix of a rooted spanning tree of G. 

If we have a minimum-cost network flow problem defined on a graph G 
having more than one component, each component having at least one root arc, 
or if the problem is defined on a graph G having only one component but more 
than one root arc, then a basis is characterized in general by a rooted spanning 
forest (see Exercise 9.37). However, as shown in Section 9.7, we can easily 
transform such a problem equivalently to a form where G is a connected digraph 
having a single root arc. Hence, Theorem 9.1 characterizes a basis matrix 
without loss of generality. 


Triangularity, Integrality, and Total Unimodularity 


For a minimum—cost network flow program defined on a digraph G having one 
root arc, we have seen that every basis B corresponds to a rooted spanning tree. 
Moreover, this basis has elements +1 or 0 and can be put in lower triangular 
form with +1 or —1 on the diagonals. This fact implies two very useful features. 
First, the systems of equations Bxg = b and wB= cz that determine the values 


of the basic variables xp and the dual variables (simplex multipliers) w can be 


efficiently solved by a simple forward or backward substitution process. We 
shall see shortly how these equations can be solved directly on the network 
itself. Second, if the b;—values are all integers, then since det B = +1 and B has 


all integer components, we see by Cramer’s Rule or by directly solving the 
triangular system that the solution to Bxpz = b is all integer. Hence, every 


extreme point is integer—valued. Therefore, if the problem requires an integer 
optimal solution, then we can solve it by simply determining an optimal extreme 
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point solution to the linear programming network flow problem obtained by 
ignoring the integrality restrictions. 

There is a richer property that the matrix A satisfies, namely, that of total 
unimodularity. A matrix A is said to be fotally unimodular if every square 
submatrix of A has determinant +1, —1, or 0. In the case of the node—arc 
incidence matrix A, since all entries are +1 or 0, every 1 x 1 submatrix has 
determinant +1 or 0. Hence, by induction, suppose that this property is true for 
every square submatrix of size (k — 1) x (k — 1), and let A, be any kx k 
submatrix of A, where k > 2. We must show that det A, = +1 or 0. Note that 
each column of A, has either all zeros or only a single nonzero entry that is +1 
or —l, or it has exactly two nonzero entries, namely a +1 and a—1. If any column 
of A, is zero, then det A; = 0. If any column of A, has a single nonzero 
entry, then expanding the determinant of A; by the minors of that column, we 
get det A, =+det A;_,, where Ajz_; is a square submatrix of size (k— 1) x (k 
— 1). By the induction hypothesis, det A,_; =+1 or 0, and hence det A, =+1 
or 0. Otherwise, every column of A, must have a +] and a —1. In this case, 
since the rows of A; add up to the zero vector, we have that det A; = 0. 


Hence, A is totally unimodular. 

Note that this proof holds true even if we delete rows or columns from A, 
or more importantly, if we add + unit vector columns to A. These columns may 
correspond to slack or surplus or artificial variable columns. The resulting 
coefficient matrix in such cases continues to be totally unimodular. Conse- 
quently, det B = +1 for all bases B, and every extreme point is integer-valued 


for integer right-hand-sides b. For any basis B, the inverse matrix B! is also 
comprised of elements that are +1 or 0 (why?). Furthermore, the updated 
column y,; in any canonical representation (simplex tableau) of a basic solution 


is comprised of +1 or 0 elements. To see this, note that y, is given by the 
system By,; = a,;. Hence, the Ath element y,, of y,; is given by Cramer’s Rule as 


= det B;, 
Tie det B ’ 


where B, is obtained from B by replacing its Ath column with a,;. Hence, B, 


is a square submatrix of A, and so det B, = +1 or 0. Because det B = +1, it 
follows that yj, = +1 or 0. Hence, any column a, can be obtained by the 
simple addition and subtraction of basic variable columns. (We shall also see 
this fact constructively later.) 


To summarize, we have shown that the matrix A°, comprised of the 
node-arc incidence matrix of a graph along with + unit vector columns, is 
totally unimodular. Hence, for any basis B of A°, B and B™ have elements 


that are +1 or 0, det B = det B! = +1, the extreme points of {x : A°x = b, x > 
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0} are all integer—valued for any integer vector b, and B'A® has all +1 or 0 


elements. 


9.4 REPRESENTATION OF A NONBASIC VECTOR IN TERMS OF 
THE BASIC VECTORS 


Consider the basis subgraph Gg corresponding to a rooted spanning tree; select 
any nonbasic arc (p, g). Because Gp is a tree, we know that there is a unique 


chain between nodes p and q. This chain is called a basis equivalent chain, and 
together with the nonbasic arc (p, q), constitutes a cycle (see Figure 9.7). 
Assigning the cycle an orientation consistent with (p, g), we have 


Ang ~ App tay t--ta 


Pq qt 


=(€, —€,)—(e, —€;) +(e —€;) + +(e, —e,) =0 


or 
ag tea 


This development leads to the following simple procedure for represent- 
ing any nonbasic column in terms of the basic columns. First, the unique cycle 
formed by introducing the nonbasic arc into the basis subgraph is determined. 
The cycle is then given an orientation consistent with the nonbasic variable. A 
basic column in the cycle that has the same orientation receives a coefficient of 
—1 in the representation, and a basic column in the cycle opposite to its 
orientation receives a coefficient of +1 in the representation. Other basic 
columns receive zero coefficients. 

As an example, consider the subgraph of Figure 9.5, which is a basis for 
the network of Figure 9.4. Suppose that we seek the representation of the 
nonbasic arc (1, 2). Using the foregoing rule, we get 


Orientation ion 


eas 


Figure 9.7. Cycle formed by adding a nonbasic arc to the basis tree. 
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Aj2 = Aj3 —A3 = (e; —€3)—(€2 —€3) =e —€. 


Note that the coefficients in the representation of the nonbasic column 


A pg in terms of the basic columns give rise to the vector y pg [that is, they yield 


Pq 
the entries in the simplex tableau under the (py, g) column]. Because the artificial 
column never appears in the representation for any other column, and because 
the artificial variable always remains basic at value zero, we may select any 
value for its associated cost coefficient, say, c, = 0. 


9.5 THE SIMPLEX METHOD FOR NETWORK FLOW PROBLEMS 


The general steps of the simplex method are as follows. First, find a starting 
basic feasible solution. Next, compute z; —c, for each nonbasic variable x ;. If 


optimality is achieved, stop; otherwise, select an entering column. If 
unboundedness is not achieved, determine the exiting (blocking) column and 
pivot. The following paragraphs present a discussion of each of these operations 
applied to network flow problems. For the moment, we shall postpone the 
difficulties associated with identifying a feasible basis (that is, Phase I of the 
simplex method) and assume that a feasible basis is at hand. We shall also 
assume that the problem is defined on a digraph having one root arc. To fix 
ideas, we shall apply each of the foregoing steps to the problem presented in 
Figure 9.8. 


Computing the Values of the Basic Variables 


Adding the artificial arc to node 5, suppose that we select the feasible 
basis given by the subgraph in Figure 9.9. The basic system of equations Bx z = 
b to be solved is 


where xs is the artificial variable associated with node 5. 


Taking advantage of the lower triangular structure of the basis matrix, we 
may iteratively solve for the basic variables. From the top equation, x,5 = 2. 


From the second equation, x»; = 5. From the third equation, x34 =1+ x3 =6. 
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bs =-4 


Figure 9.8. An example network flow problem. 
by =5 


Figure 9.9. A basis subgraph. 


Next, x45 =—4+ x34 = 2. Finally, x; =-4+ x;5; + x45 = 0 and thus, the basis 
is feasible. These same computations can be made directly on the graph in 
Figure 9.9 as follows: 

Examining node | in Figure 9.9, we see that it is an end of the basis tree, 
that is, it is a node having only one basic arc incident to it. Hence, the corre- 
sponding basic equation contains only one variable, and the value of that 
variable can be readily obtained. In this case, arc (1, 5) points out of node 1 and 
thus, x,5; has a+1 in row 1. Therefore, x5 =b,, or x15 =2. 


45 = 
b= <_—>, Xs =2. 
*15 


Examining node 2, we see that it is an end node and hence, x23 can be 
computed similarly: 
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X93 = by 


X73 —_—_ X93 =5. 


Next, notice that node 3 has values assigned for all of its incident arc variables, 
except one. Thus, we can use the conservation equation for node 3 to solve for 
this remaining variable: 


X73 =5 
——s X34 — X23 =1 
X54 X34 = 6. 
b =1 
We can now solve for x45: 
xX 4 = 6 
casita b,=-4 X45 —%34 =—4 
Xgs =2 
X45 
Finally, we solve for x5: 
X45 = 2 
Xj5 =2 co 
bs =-4 Xs —%y5 — X45 =—4 
Xs = 0. 


x5 


The process of obtaining the basic solution proceeds from the ends of the 
tree toward the root (see Figure 9.10). As we shall see later, the process of 
obtaining the dual variables is just reversed. 


End End 


Root & 


Figure 9.10. Computing the values of the basic variables. 
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Let us provide an insight with respect to the computation of y,, via the 


system By ,, = in Section 9.4. Note that this is similar to the system “Bx, 


apg 


= b,” where a,, plays the role of the vector “b.” Since a,, =e, —€,, the 


e 
pq *p” *q 
situation is analogous to having a supply of | at node p and a demand of | (net 
supply of —1) at node q. All other nodes are transshipment nodes. Hence, the 


solution to BY pg = apg corresponds to sending a flow of 1 along the basis 


equivalent chain from p to g. Consequently, referring to Figure 9.7, the coeffi- 
cients of y,, corresponding to the basic variables x,;, xz¢, and x,, are +1, and 


the coefficients corresponding to the basic variables x,; and xg, are —1. 


Computing Dual Variables w and 2 — cj 


Given a basis subgraph, we need to compute Zz; — Cy for each nonbasic variable 
x, and either stop or proceed by entering a nonbasic variable having a positive 
2y — Cy. Toward this end, we may compute the dual or simplex multiplier vector 


w through the system wB = cz, and then determine z;—c, through the 


ij 
expression Zy — Cy = Way —c, . For the basis subgraph of Figure 9.9 we have, 
1 0 0 00 
0 1 0 00 
[ wy,W2,03,W4,Ws ]] O -1 1 0 0 |=[2,-4,0,3,0]. 
0 0-1 10 
-1 0 0-1 1 


Note that the rows of the lower triangular basis matrix are also permuted 
as 1, 2,3, 4, and S. 

Using the last w equation, the one associated with the root arc, we get ws 
= 0. We may now proceed away from the root in the following fashion: 


Wg — Ws = C45 => Wy =34+0=3 
W3— Wg = C34 > W3 =04+3=3 
W7 —W3 = C93 > WW, =-44+3=-1 
Wi - Ws =Cy5 DW =24+0=2. 


We therefore start by setting the dual variable for the root node at zero 
value, and then proceed away from the root node toward the ends of the tree 


using the relationship that w; —w, = cj along the basic arcs in the tree. 


While the process of computing primal variables consisted of working 
from the ends of the basis tree inward toward the root node (see Figure 9.10), 
the process of computing dual variables consists of working from the root node 
of the basis tree outward toward the ends (see Figure 9.11). 

To compute z;;—cj for the nonbasic arc (i, /), we apply the definition 
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Root 
Figure 9.11. Computing the values of the dual variables. 


Zij 7 oy = wa; mea 


we; =O) Cy 


= WwW; —W; — Cy. 


Thus, the zj—cj can be conveniently computed on the network. Notice also 
that by requiring that w; -w; =cj along basic arcs, we are actually requiring 
that z,; —-cj = 0 for basic variables. 


Using the values of the dual variables obtained earlier, we summarize 
below the value of zj; —cj for each nonbasic variable x;. 


Let us provide some interpretations for the dual variables w; and for the 


reduced costs cj —zj. The dual variables w; are sometimes referred to as node 


potentials because of the following analogy. Consider a basis tree and suppose 
that the flow sustained by this tree has resulted in certain pressure heads or 
“node potentials” at the nodes. Imagine that the cj values are reverse pressures 


or forces along the arcs and that the root node is at a zero potential. The equations 
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w; —w, =cj for the basic arcs assert that the net force w; — w; along (i, /) due to 


the node potential difference equals the back pressure c;;. Hence, the system wB 


= ¢p determines, as if it were, the node potentials w at equilibrium or steady 


state, with respect to the flow sustained by the basis tree. For a nonbasic variable 


arc (i, /), if w; —w 7 ey SO, then this says that the back pressure Cj is at least 


as much as the potential difference w,—w ' along (i, /), and so the net flow 
pressure along (i, /) is nonpositive. If this is true for all nonbasic arcs, then the 
entire network is in equilibrium, and we are optimal. On the other hand, if 


w; —w; —cj >0, then the node potential difference w;—w, exceeds the back 


pressure cj. This will have the tendency to force flow along (i, 7). Hence, we 
can enter xj into the basis in this case. 

Observe that only the differentials, rather than the particular values, of the 
w-variables matter in the foregoing analysis. Indeed, since the cost c, on the 
root arc can be fixed at any arbitrary value, if w represents the dual variables 
calculated with c, = 0, then with an arbitrary cost value c,, the w—values are 
given by w; =,+c, for all i = 1,..., m. This is evident from the nature of the 
system WB = Cz (why?). Hence, all the dual variables can be scaled up or down 
by a constant if desired. 

The node potentials have an interesting marginal cost economic inter- 
pretation as well. Recall that w; = 0z/0b; in the usual sense, that is, w; is the 
increase in cost if b; is increased (marginally) by a unit and if all other right 
hand-sides and nonbasic variables are held at their current values. But 
increasing 6; by a unit amounts to putting an extra unit of supply at node i, 


which must consequently leak out through the root arc via the unique chain 
joining node i to the root node because of the conservation of flow equations. 
Hence, w; is the cost ofa unit flow along this chain. (Note that some arcs in this 
chain may have a reverse orientation, and so the flow on that arc would decrease 
by a unit.) For example, for node 2 in Figure 9.9, wy =c 3 +¢34 +¢q5 =—1, and 


for node & in Figure 9.11, wy = cg — cg + Cpp- 


Let us now define z; =—w; for i= 1,...,m, so that ; is interpreted as the 


increase in cost if an extra unit of demand is created at node i (given that flow 
can occur only on the arcs in the basis tree). Hence, if each node is imagined to 
be a market, z; plays the role of the marginal cost of supplying an extra unit of 
the product at market i. (By using c, as previously, we can make all z; = 0 if 
so desired.) For any basic arc (i, /), the equation w, WwW, = Cy 
that is, we break even if we purchase a unit at market i, transport it 


implies that 


ay Sj t+ Cy, 


along the arc (i, /), and sell it at market j. Similarly, if w; —w,;—c¢y <0 for any 
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nonbasic arc (i, /), then 7; <7; + ey; therefore, the foregoing venture is not 


profitable. If this is true for all nonbasic arcs, then the market system is in 
equilibrium—the current set of basic flows have generated a set of prices that 


hold the flows in steady state. On the other hand, if w, — Wi Cy >0 for a 


j > % +¢j, and an entrepreneur would find it profit- 


able to purchase in market i and sell at market j by transporting the product over 
the arc (i, 7). For example, for the nonbasic are (1, 3), 2;3-—cj3 = W— W3- 


nonbasic arc (i, /), then z 


c13 =1, and so it is profitable to pay z,=-—2 per unit at node 1, pay the 
transportation cost of c}; =—2 per unit, and receive 73 =-—3 per unit at node 3. 
The net gain is {—2) — (-2) + (-3) = 1 per unit. 

Note also that Zy Cy = ORY gy — Cy- From Section 9.4, we see that Yi has 


a +1 corresponding to basic arcs that point in a direction opposite to the 
orientation of flow change in the cycle created by introducing (i, /) in the basis, 
a —] corresponding to basic arcs in the cycle that have the same orientation as 


the flow change, and zero otherwise. Hence, Zi — Cy is the negative of the cost 


of sending a unit along the cycle in the same orientation as (i, /). For example, 
213 —C13 = C34 —Cq5 + C15 —Cy3 =9-34+2-(-2)=1. Therefore, the objective 
function will fall by a unit for every unit of flow sent around this cycle in the 
orientation of (i, j). Although this gives another method of computing z,; — Cy; it 
is less efficient than the earlier method of computing Zz —¢j through the dual 


variables w (why?). Hence, it is not used in practice. 


Determining the Exiting Column and Pivoting 


When we applied the cycle method to compute z,;—c;; for a nonbasic arc, we 


essentially identified the pivot process. In the foregoing example, z,3—c,3 > 0 
and so, x,3 1s a candidate to enter the basis. What we must do is proceed to 
increase x,3, adjust the basic variables to maintain feasibility with respect to the 


right—hand-side, and determine the first basic variable (if any) to reach zero. 
This blocking basic variable, if it exists, is the exiting variable that leaves the 
basis. 

Consider the basis tree together with arc (1, 3). If we increase x3 by A, 


then to provide balance, we must increase x34 by A, increase x45 by A, and 
finally decrease x,5 by A. This process of adjustment can be thought of as 


sending an additional amount of flow A around the unique cycle created when 
the nonbasic arc is added to the basis tree. Naturally, sending flow against the 
direction of an arc corresponds to decreasing flow on the arc. 
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As x;3 increases by A, the only basic variable to decrease is x,5 and its 
new value is x,5 = 2-—- A. Thus, the critical value of A is equal to 2, at which 
instant x5 drops to zero and leaves the basis. All of the other basic variables 


are adjusted appropriately in value and the new basic solution is given as 
follows: 


( ’ X43 =2 


We leave it as an exercise for the reader to show that determining the exiting 
variable, and adjusting the values of the basic variables accordingly as described 
earlier, is equivalent to performing the usual minimum ratio test and pivoting. 
We have seen earlier how the basis tree and the flows are updated in a 
pivot operation. To complete the discussion of an iteration of the network 
simplex method, let us show how the dual variables may be updated rather than 
recomputed from scratch with respect to the new basis tree. Suppose that x,, 


enters the basis, and x,,, is the exiting variable. Disconnect the leaving arc (u, v) 


from the current basis tree. By Property 1 of tree graphs, this decomposes the 
tree into two trees 7; and 7>, with the root node r, say, belonging to tree 7). 


Hence, the new dual variable values for the nodes in 7; will remain the same as 


before, since the chains connecting these nodes to the root node remained 
unchanged. For tree 7>, let us consider two cases. First suppose that q €T7). 
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Node q is then called the leading node with respect to the entering arc (p, q). 


Note that for each (i, /) € 7), we currently have w; — w; =cj. If we change all 


the w;-values in 7, by a constant, we will still satisfy w;-w,; =c, for all 


(i, /) € Ty. Consequently, once we know the new value Wo(pew) of the dual 
variable associated with node g, we can compute the new dual value Winew) for 
each node i in Ty as w; + (Wa(new) — Wg)» Since (Wo(new) — Wg) is the amount by 


which w, has increased. However, denoting 6,4 =2Zpg ~Cp 
Ong =Wp — Wg -Cpgs While Wy -Wo(new) =Cpq (Why?). Hence, wo(new) — 


Wg = Opq> 


On the other hand, if p is the leading node, that is, p € T>, then we have 


g >0, we have 


and all the dual variables in 7, simply increase by 6,,,. 


Wo(new) —%q = pg ANd Wanew) — Wp = —Opq- Hence, in this case, all the duals 


of 7, will remain the same as previously, but the duals of 7) will fall by dp,. 


For example, in the foregoing pivot operation, (p, g) = (1, 3) enters and (A, @) = 
(1, 5) leaves the basis. Disconnecting (1, 5), we find that the tree 7, contains the 


nodes 2, 3, 4, and 5, but 7, contains node 1 alone. Since p €7>, the dual of 
node | falls by 6,, = 1 to the value wi(new) = | and the duals of the other nodes 


remain the same as before. 


Summary of the Network Simplex Algorithm 


INITIALIZATION STEP 


Find an initial basic feasible solution represented by a rooted spanning tree, with 
r as the root node. (Section 9.7 discusses how to achieve this, using artificial 
variables if necessary.) Compute the basic flows xg and the dual variables w 


associated with this basis tree. 


MAIN STEP 


Compute Ong =2 pg —Cpg =Wp — Wg ~Cpg = Maximum {2 —cy =w;,—-w;—- ¢, 


a <pq ~“p —%q ~ © pg i (hs 
(i, /) is nonbasic}. If z pq ~ pq £9, then stop; the current solution is optimal. (If 


any artificial variables are positive at optimality, the problem is infeasible.) 
Otherwise, introduce (p, qg) into the basis tree and determine the unique cycle 
formed by tracing the basis equivalent chain connecting p and q. Find the 
maximum flow A> 0 that can be sent along this cycle in the orientation of the 
entering arc (p, g). If A > ©, then stop; the problem is unbounded. (If the big— 
M method is being used and some artificial variable is positive, the original 
problem is infeasible; otherwise, it is unbounded.) If A is finite, determine an 
exiting or blocking variable arc (u, v). Update flows by appropriately adjusting 
the flows in the cycle by A. Disconnect (uw, v) from the basis tree and decompose 
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it into two trees 7, and 7,, where the root node r € 7,. Update the basis tree by 


adding arc (p, q) to it. Update the dual variables by respectively increasing or 


decreasing the dual values of nodes in T, by 6 oie according to whether g or p is 


the (leading) node in 7>. Repeat the main step. 
9.6 AN EXAMPLE OF THE NETWORK SIMPLEX METHOD 


Example 9.1 


Consider the network of Figure 9.12. In Figure 9.13 we describe the complete 
solution of this minimal cost network flow problem. The value 6 pq = 2pq ~<pq 
for the entering variable is circled. The exiting variable is denoted by * in the 
figure. 


9.7 FINDING AN INITIAL BASIC FEASIBLE SOLUTION 


We have assumed thus far that we are given a minimum cost network flow 
problem defined on a connected digraph having a single root arc and that we 
have a starting basic feasible solution represented by a rooted spanning tree to 
commence the network simplex algorithm. We now give a method for generally 
attaining this situation. 

Suppose that by adding any necessary slack or surplus variables and by 
employing the usual transformations on variables (including the replacement of 
undirected arcs that have nonnegative costs by two oppositely oriented directed 


arcs), we have put the given problem in the form: Minimize {ex : A°x=b,x> 


0}, where A° is mx nand is composed of columns that are either + unit vector 
columns or that have a +1 in some row, a —1 in some other row, and zeros 
elsewhere. Hence, we have a network structured problem. Although we can 


possibly use some of the arcs (columns) in A° itself as part of a starting basis, 
let us present the general use of an all artificial start. 


Suppose that we add an artificial column for every row of A®, the ith 
artificial column being +e; depending on the sign of 5; (that is, +e; if b; = 0; 
—e; otherwise). Also, let us add a redundant row given by the negative of the sum 


Figure 9.12. An example network. 
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Iteration Primal solution Dual solution 25 — Cy Pivot 


Optimal 


x13 =4, x93 =2, x34 =5, allother x; =0, 2" =-7. 
Figure 9.13. The solution to Example 9.1. 


of the rows of the “extended” A° matrix. The problem constraints then become 
as follows, where x, is a vector of artificial variables: 


Let the foregoing system be denoted by Asa = Dnew. Observe that each 


column in ee has exactly one +1, one —1, and zeros elsewhere. Hence, it may 
be viewed as a graph. This new graph has one additional node (m + 1) because 


of the new row in As: Furthermore, all the original arcs in A° are present in 


AO as well, with the unit slack (root) arc columns in A° either terminating or 


Minimal Cost Network Flows 477 


originating at the new node (m + 1) in Ale. Additionally, the new graph has m 


new arcs—one arc between each original node and the new node. Observe that 
because of the artificial arcs, the graph is clearly connected, and moreover, the 
elements of Dyey add to zero as they should. Furthermore, arbitrarily 


designating the new node (m + 1) as the root node r and adding the unit column 
€(m+1) tO AP... , which acts as the single root arc in the problem, we obtain a full 


row rank system. A feasible basis for this new problem is given by that rooted 
spanning tree that is defined by the m artificial variable arcs in addition to the 
root arc. 

Beginning with this artificial basis, we may proceed to apply the two— 
phase method or the big—M method, using appropriate costs in each case, until 
feasibility is achieved, if at all. If feasibility is achieved, we may drop all of the 
artificial arcs not in the basis, keep the basic degenerate artificial arcs with a 
big—M cost in the problem, and continue the optimization. 

To illustrate the technique, consider Example 9.1. After adding 
appropriate artificial columns and creating the new row, we get the following: 


M213 X34 32 BH HS brew 


MAP WN 


Selecting the two-phase method, the artificial variables x,, x2, x3, and x4 in 
Phase I will have cost coefficients of | while all other variables have zero cost 
coefficients. This leads to the associated network flow problem of Figure 9.14 
where the cost coefficient of the root x5 is zero. With this feasible basis at hand 


we proceed to solve the Phase I problem, using the procedures developed in this 
chapter. 


The Phase | problem A feasible basis 
Figure 9.14. The Phase I network flow problem and a starting 
feasible basis. 
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Example 9.2 
As another example, consider the following problem: 
Minimize xj2 + 2x)3 + 3x93 -— 2x94 + 42x34 
subjectto x2 + X43 < 6 
—x\2 + X23 + X24 < 4 
— %130 — X23 + x34 S -2 
ag. Mg S35 
X12 X13> X35 X24, x34 = 02. 


This problem states that no more than six and four units of supply are available 
at nodes | and 2, respectively. At least two and five units of demand also must 
be absorbed at nodes 3 and 4, respectively, with additional units being 
permissible if this is profitable. We now add slack variables s), sz, 53, and s4 
with zero costs and add appropriate artificial variables x, x2, x3, and x4 with 
big—M costs to the problem. Creating a new row equal to the negative of the sum 
of the other rows and including the artificial root~arc variable x; with a zero 
cost yields the following system of equations: 


*12_ 13X23, X24 34 


We now ask the reader to draw the graph corresponding to this system, to 
construct the starting artificial basis tree, and to optimize using the big-M 
method (see Exercise 9.14). 

Note also that we could have used the slacks s, and sz in lieu of the artifi- 
cial variables x, and x», respectively in the starting basis. This use of an all- 
artificial start is only for illustrative purposes. 


9.8 NETWORK FLOWS WITH LOWER AND UPPER BOUNDS 


It is simple and straightforward to make the transition from the ordinary simplex 
method for network flow problems to the bounded variables simplex method for 
network flow problems. We briefly review the essential changes required to 
effect such an extension to this method. 


Getting Started and Computing the Flows 


Via standard transformations, including the method of Section 9.7, we can 
assume without loss of generality that the given problem is in the form: 


Minimize {ex : A°x =b£<x< u} 
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where A° is an (m + 1) x m node-arc incidence matrix of a connected digraph, 
including m appropriate artificial arcs and a single root arc, >',b, = 0, and where 


—0 < fi <ujy <0 for all arcs (i,j) in the problem. Using the Phase I or the big— 


M method, all the original arcs in the problem may be set at one of their (finite) 
bounds for the initial basis. This uniquely determines the flow values of the m 
artificial variables, and uniquely determines the root—arc flow value as zero 
(why?). (Note that the artificial variables should be added with the appropriate 
signs so that they are all nonnegative in this solution.) Hence, we have a starting 
basic feasible solution. For the remaining iterations, we need not worry about 
the conservation of flow equations, since we will automatically enforce these 
equations by modifying the flow in cycles. This process will also automatically 
update the arc flows. However, if we are given a basic partition and we need to 
compute the flows, this may be done by adjusting the right—-hand-side values to 
b -N)£y,; —N2uy> (in the notation of Section 5.2) and solving the system 


Bx, =b- Nien —-Nouy? (as in Section 9.5). 


Computing the Dual Variables and the Zi — cy values 
Lower and upper bounds have no effect on the computation of the dual variables 
and on the computation of the Zy — Cy values. Note, however, that in the 


presence of lower and upper bounds the optimality criteria are 


=u; > z;,-c,; 20 


yp Aye aap Gy 


and 
These are easy to check and we can readily determine whether some nonbasic 
variable x;; should be increased or decreased if optimality is not achieved. 


Determining the Exiting Column and Pivoting 


Once the entering column is selected, it is again an easy task to select the exiting 
column and to pivot. We add the entering nonbasic arc, regardless of whether 
the variable is increasing or decreasing, to the basis tree and determine the 
unique cycle formed. Then, if the entering variable is increasing, we send an 
amount A around the cycle in the direction of the entering variable. If the 
entering variable is decreasing, we send an amount A around the cycle against 
the direction of the entering variable. Figure 9.15 illustrates these two 
possibilities. To compute the critical value of A we check those basic variables 
increasing as well as those decreasing and the possibility that x;; may reach its 


other bound. If the last possibility occurs, xj; remains nonbasic (at its other 


bound) and all basic variables along the cycle are adjusted accordingly. 
Otherwise, the nonbasic variable enters and some basic variable exits at one or 
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Figure 9.15. Two cases for entering are: (a) x, increasing; 
(b) Xj decreasing. 
other of its bounds, and all variables along the cycle are adjusted accordingly. If 


the basis tree remains the same, the duals are unchanged. Otherwise, the duals 
are updated exactly as before. 


An Example of the Network Simplex Method in the Presence of Lower and 
Upper Bounds 


Consider the network flow problem of Figure 9.16. We present, in Figure 9.17, 
the complete solution to this problem. We have identified a starting basic 


feasible solution, thus omitting Phase 1. The notation i ae represents a 


nonbasic arc at value w. The exiting variable is noted by *. 


Table 9.1 The Simplex Tableau Associated with the Final Basis in Figure 
9.13. 


2 %12 =—%13—*23—— 24 = X32 X34 = Gi. 


b =3 by =-8 


Figure 9.16. An example network with lower and upper bounds. 
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Iteration Primal solution Dual solution Z;; — C3 Pivot 


1 3 / }4 / -2 O 
‘ 6 0 O 
O 


A=min{3, 4, 1} =1 


i 
10 9 
O 
a yu | 28 [+7 Optimal 
0 es 
Ve. Rae O16 
6 0 


* * * * * * 
Xp =3, X43 =2, Xg =4, XD =1, Hq = 4, z = 42. 


Figure 9.17. The solution to the example in Figure 9.16. 


9.9 THE SIMPLEX TABLEAU ASSOCIATED WITH A NETWORK 
FLOW PROBLEM 


In Section 9.4, we showed how to construct the column yj, for any nonbasic arc 


(i, /). Elsewhere, we have seen how to obtain the values of the basic variables 
and the z-—cj values. Thus, it is possible to construct the entire updated 
tableau by examining the basis subgraph at the corresponding iteration. 

As an example, consider the final (optimal) basis in Figure 9.13 for the 
network flow problem of Example 9.1. The simplex tableau for this basis is 
given in Table 9.1, where x4, denotes the artificial variable at node 4. To 
illustrate how a particular nonbasic column is obtained, consider x,2. We have 
already indicated how z,;7 —cj2 =—6 may be computed. To produce the other 


entries in the column we consider the unique chain formed by adding (1, 2) to 
the basis subgraph. The unique chain in the basis tree is C = {(1, 3), (2, 3)}. To 
reorient this into a path from 1 to 2 we multiply column x,3 by 1 and x3 by —1; 


thus we obtain the coefficients in the tableau. As a check, we see that 


a}3 423 = (€; —€3)—(e2 —€3) 
Ceo 412 


as required: 


The other columns of the simplex tableau are obtained in a similar manner. 
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9.10. LIST STRUCTURES FOR IMPLEMENTING THE NETWORK 
SIMPLEX ALGORITHM 


In this section, we will describe an enhancement of the augmented threaded 
index (ATI) list structures for efficiently implementing the network simplex 
algorithm on the computer. Although simpler labeling techniques are possible, 
the method we describe is very efficient. In this process, we shall also introduce 
terminology and concepts that are now standard in network programming 
literature. 

The main construct in any list structure scheme lies in the representation 
of a rooted spanning tree. The ATI method prescribes three fundamental node— 
length arrays that not only represent the layout of the tree, but also facilitate the 
efficient execution of a simplex iteration, and the updating of the primal and 
dual solutions and of the list structures. Additionally, three other node—length 
arrays obtainable from these three fundamental lists may be explicitly main- 
tained in order to enhance the updating of the list structures. These are described 
later along with the simplex algorithm and the updating operations. 


List Structures: Definitions 


Imagine that a rooted spanning tree T having m nodes is given, where r is the 
root node, and that this tree is laid out in the convenient form depicted in Figure 
9.18. The list structures described here enable the computer to “see” this tree in 
the form shown in the figure. The first list is the predecessor index list, denoted 
p(i), i= 1,..., m. For any node i, p(i) uniquely gives the node connected to node i 
on the chain from i to the root node. For this reason, p(i) is also referred to as the 
“down” node of i. In Figure 9.18, for example, p(1) = r, p(9) = 5, p(17) = 12, 
and p(27) = 23. Thus, p(i) precedes i or is the “parent” of i in the “growth” of 
the tree. By convention, we let p(r) = 0. 

Define the level of node i as the number of arcs in the (unique) chain 
connecting node i to the root node. Denote this by (i), i = 1,...,m. Hence, £(1) = 
1, 210) = 4, €(17) = 5S, and ¢(4) = 6 for nodes i = 19, 20,..., 25, that is, these 
nodes are all at “level” 6 with respect to the tree in Figure 9.18. 

Next, let us define the thread index t(i) for i = 1,...,m. Imagine the process 
of sewing the nodes of the tree in Figure 9.18 with a needle and a thread; 
proceeding from bottom to top, from left to right, and finally, returning to node r 
after sewing all the nodes together. The nodes might then be sewn in the order 
of the following list t = {r, 1, 2, 4, 8, 14, 19, 20, 15, 5, 9, 16, 21, 22, 3, 6, 10, 11, 
7, 12, 17, 23, 26, 27, 18, 24, 25, 13, r}. Formally, let us define the subtree 
rooted at node i to be that tree 7; that is obtained by disconnecting the arc 
connecting i and p(i) and which contains the node i. For the root node r, the tree 
T, is the entire tree 7. For example, 7g is obtained in Figure 9.18 by 


disconnecting the arc (8,4) and is the subgraph of T that is induced by the nodes 
8, 14, 15, 19, and 20. The nodes in 7; (other than node 7) are called the 


successors of i, since node i lies on the chains joining each of these nodes to the 
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Entering arc 


Figure 9.18. List structures for network flow programming. 


root node. The immediate successors of i have node i as their predecessor. We 
now can define t as an ordering of the nodes such that for any node i, the list of 
its successors, that is, the nodes in the subtree rooted at i, follow it consecutively 
in the list. Note that the ordering of the nodes in T is not unique. For example, by 
interchanging the relative positions of nodes 2 and 3 in Figure 9.18 and 
following the bottom-top, left-right rule, we would have obtained another valid 
ordering. In any case, the ordering should be such that once a node i is “sewn,” 
the next set of nodes sewn are the successors of i. The function ¢(i), also known 
as the next node of i, is then identified with t according to t = {r, ¢(r), «(t(r)) = 
(nr), O(n)... 1" |, e%() =r}, where t* (rn) composes ¢(-) with itself k times, 1 
<k<m. Hence, t(r) = 1, t(1) = 2, #(2) = 4,..., (24) = 25, (25) = 13, and #(13) = 
r. Note that the nodes in 7; are readily obtained using the lists ¢(-) and £(-). 


Namely, 7; is comprised of nodes (i) = 1, ti), P(A), tf (i), where k is the 


largest positive integer such that ¢{t/(i)] > ¢(i) for j = 1.,...,k, if it exists, and is 
zero otherwise. 
For convenience, a reverse thread index tp(-) is also sometimes main- 


tained, where tp(i) =, if and only if «¥) = i. By way of terminology, the nodes 


that come before node / in the list t are called its antecedents, and the preorder 
distance of i # r is its rank or position in the list t. The traversal of the tree 
following the thread index is called a preorder traversal, and a traversal 
following the reverse thread is called a postorder traversal. Additionally, 
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another useful list both conceptually as well as computationally is fi), the final 
of node i for i= |,...,m. Given that 7; is comprised of the nodes (i) = i,...t* (, 


k> 0, the node fi) = (i) is the final node threaded in 7;. Loosely speaking, fli) 


is the “final or last successor’ of i in the list t. Note that if 7 is an end node, then 
7; =i and fi) = i. Let us also denote another list n(i) as the number of nodes in 


T,, i=1,...,m. Thus, fi) = "OU, In Figure 9.18, for example, (8) = 15, n(8) 


= 5, A9) = 22, n(9) = 4, A7) = 13, n(7) = 10, A21) = 21, n(21) = 1, Kr) = 13, and 
n(r) = 28. 


Initialization 


The foregoing lists are readily constructed for the all artificial basis in which the 
tree is comprised of arcs connecting each node and the root node. For this tree, 
we have p(i) =r for all i# r and p(r) = 0, 4) = 1 for all i#r, and &(r) = 0, t(r) = 
1, (1) = 2, (2) = 3.,...,40m — 1) = r (assuming that node “m” is the root r). Also, 
AO =i and n(i) = 1 for all i# r, while fr) = m— 1 and n(r) = m. 

On the other hand, if a certain advanced basis is available, then these lists 
may be initialized as follows. (Use the example of Figure 9.18 while reading 
this procedure.) Pick some node as the root node r. Let S(r) be the immediate 
successors of r determined by finding the basic arcs incident at r. Then, p(i) =r 
and (i) = 1 for all ie S(r). Pick some node je S(r), put ¢(r) =j, and remove j 
from S(r). Similarly, construct S(j) as the set of all nodes other than node r that 
are adjacent to node j for some basic arc. Assume that S(/) # ©. Then p(i) =j 
and £(i) = £4) + 1 for all ie S(j). Pick some node ke S(j), put «(/) = k, and 


remove k from S (/). Continue in this manner until some node e is reached for 
which S(e) = ©. Hence, e is an end node of the basis tree T. In this case, follow 
the predecessor index to come down the tree until a node g is reached for which 
S(q) # ©. In this event, pick some £€ S(qg), put t(e) = 2, and remove ¢ from 
S(q). While coming down the tree, each node i that is crossed over downwards, 
including node e itself, gets node e as its final successor node fi). Also, the 
number of new nodes labeled between threading i and giving node i its f(-}- 
value yields n(i). The process can now be continued by constructing S(¢) and 
proceeding “up” the tree again. The procedure stops at the point when following 
the chain down from some end e, it reaches node r and finds S(r) = ©, 
whereupon, in particular, among other labels, it sets fr) = e and t(e) =r. 


Simplex Operations 


Observe that each arc in the problem is uniquely identified with some node via 
its predecessor index. That is, for each node i # r, there is an associated, 
uniquely identified arc connecting i and p(i). This arc may be [i, p(i)] or [p(a), 2]. 
Hence, a node—length array ORIENT(-) may be carried to record the orientation 
of the basic arcs, having a + | in one case and a — | in the other case, 
respectively. For example, for node 20, the associated arc is [p(20), 20] = (14, 20) 
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and ORIENT(20) = -1, while for node 8, the associated arc is [8, p(8)] = (8, 4) 
and ORIENT(8) = 1. Consequently, the basic arc flow vector can be stored in a 
node-—length array FLOW(-) , where FLOW(i) is the flow on the arc connecting 7 
and p(i). Let DUAL(é) represent the dual variable associated with node i. The 
arrays DUAL(.) and FLOW(.) are readily initialized via a preorder traversal and 
a postorder traversal, respectively. For example, in Figure 9.18, the dual values 
may be determined in the order of the nodes r, 1, 2, 4, 8, 14, 19, 20, 15,... as 
they appear in T, based on the arc connecting each node i and its predecessor 
p(i), since the dual of p(i) is already computed for each i in this order. Similarly, 
the flows may be computed by following the nodes i in the reverse thread order 
13, 25, 24, 18, 27, 26, 23, 17, 12,..., and determining the flow on the arc 
connecting i and p(i). This is based on the nodal balance equation for node i in 
this order, since the flow on at most one incident arc is unknown in each case. 
Suppose that by following the usual pricing scheme, we determine an 
entering arc (p, qg). The basis equivalent chain may be traced by coming down 
from p and g following the predecessor index list p(-) until a first intersection 
point & is reached. Note that node k may be p or q itself. Hence, if 4(p) > 4(q), 
say, then we can come down from p until we reach the level of g, from where it 
is known whether or not & = gq. If not, then we can come down simultaneously 
one level at a time along the two chains until they intersect. While doing this, a 
minimum ratio test can be conducted simultaneously (how?), so that the leaving 
arc is known once the basis equivalent chain has been traced. Let the leaving arc 
connect nodes u and v, where u = p(v) (see Figure 9.18). Note that the actual 
leaving arc may be (u, v) or (v, u). When the leaving arc is disconnected from T, 
the tree not containing the root is the subtree J, . Let us also assume that q is the 


leading node in 7,—the entering arc may actually be (p, q) or (q, p). 


Updating Operations 


The flows are updated as usual on the basis equivalent chain by tracing down 
from p and q to k after determining the flow change A. The predecessor index 
p(-) changes only on the g — v chain, also known as the stem, simply reversing 
or inverting on this chain with p(q) = p. Hence, in Figure 9.18, we get p(v) = 12, 
p(12) = 17, and p(17) = 8, with p(Z) as before, otherwise. Accordingly, the tree 
arcs that are associated with the nodes on the stem q — v through the predecessor 
index change in this reversal process, and so the ORIENT(.) and FLOW(-) vectors 
should be appropriately updated. For example, FLOW(12) now corresponds to 
the flow on arc (12, 17) and ORIENT(12) = 1. The dual values in T— 7, remain 


the same, while the dual values in 7, all increase by Ong = Zpq —€pq if (B, 9) is 


the entering arc, and decrease by Sap if (g, p) is the entering arc. 


=Z,n —C 

4p 9p 
Hence, the dual values can be updated by following the thread index from v to f 
(v). (Notice how z (v) helps in this respect.) 
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Finally, we need to update “-), £(-), f-), and x(-). This is done by a 
grafting process. Notice that 7, is, as if it were, rooted at node v, while 7; = 


T -T, is rooted at r. Suppose that we reroot 7, at q to get T,, that is, we obtain 


the list structures for the new tree T,; 


T,. Then we can simply graft J, onto 7; by connecting p and q to obtain the new 


and similarly, obtain the list structures for 


tree. In this process, let x= fp) and y = (x) in 7;, and let z = f(g) in T,.In Figure 
9.18, x= 15, y=5, and, say, z= 13. Then, because T, is a part of T, in the new 
tree, we set t(x) = q and tz) = y. Figure 9.19a illustrates this grafting process. 
Also, the levels of the nodes in T, after grafting will all increase by one plus the 
level of p in 7,. Furthermore, p and all nodes not in 7, p that had the final of node 
p as their last successor in 7; will now have z = f(g) as their last successor. 
Letting y = p(t(f(p)]) in 7,, these nodes are on the chain from p to Y, not 


including y if y # 0, and on the chain from p to r, otherwise. In Figure 9.18, we 
have y = p[t(15)] = p(S) = 2, and so, after grafting, we will have 8) = (4) = 13. 


Thread in T 


Figure 9.19. Updating process for the list structures. (a) Grafting. 
(b) Updating 7; . (c) Rerooting 7, at q. 
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Similarly, the n(-)—values for each node on the chain from p to r in 7; (inclusive) 


will increase by the number of nodes in 7]. Hence, the principal task in the 


grafting process is to update 7; and to reroot 7, at q. 
Updating 7, 


The p(-) and £(-) values remain the same. For the thread index, because the arc 
(u, v) is disconnected, the thread entering J, and the thread leaving 7, to 
reenter 7, need to be cut and reconnected as shown in Figure 9.19b. Let gand 0 
be such that ¢= tp(v) (that is, (¢) = v) and O= ¢[f(v)]. Then we put t(@) = Gin 
T,. The n(-)-values of the nodes on the chain from wu to r (inclusive) will fall by 
n(v). For updating f(-) in 7;, note that all the nodes that have f(v) as their final 
node will get the new fw) as their final node. With respect to the previous 
indices, we find y = p[t(f(v))], set 6 = flu) if fu) €T,, and let 5 = tp(v), 
otherwise. In Figure 9.18, f(v) = 13, 1(13) =r, p(r) = 0, and 6= tp(v) = 11. If y 
= 0, as it is here, then we put fi) = 6 for all i in the chain from u to r inclusive. 
Hence, we would have f(3) = f(1) =f(r) = 11 in the example. Otherwise, if y # 
0, the f(i) values change for i on the chain from wu to y, not including y, and 
become the same as 6. 


Rerooting 7, at q 


To begin with, consider the stem g — v. Without loss of generality, for clarity in 
exposition, we shall discuss this step with respect to Figure 9.19c. (Several of 
the following steps can be made more efficient by a simultaneous processing. A 
different scheme using the reverse thread function is also possible.) 

If vy = q, the updating is vacuous. Hence, suppose that v # q. For each 
node distinct from g in the stem, find its immediate successors not belonging to 
the stem. For example, consider node @ in Figure 9.19c. Noting that 7, is 


currently rooted at v, t(@) is either q, Q, or @. Say, t(@) = a. Then ¢[f(q)] is 
either q or @, say, g. Hence, t[f(q)] = @2. But then the level of ¢[f(@2)] is less 


than the level of @ and we know that we have found all the immediate 
successors of @as the nodes a), g, and @. 


Next, update ¢(-) as follows. Leave the ¢(.)-values unchanged for nodes i 
in the current subtree rooted at g, except put ¢[f(q)] = a. Now, put ¢(@) = a, and 


leave ¢(é) the same for nodes i in the subtree rooted at a, except put ¢[f( a )] = 
Q@. Again, leave 1(i) the same in the subtree rooted at a, except put t[f(az)] = 


B. (Notice how the use of n(-) makes this process efficient.) Hence, we essen- 
tially tie up the loose thread ends shown in Figure 9.19c until finally, we put 


tf(v2)] = 4. 
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The levels &(-) of the nodes that are shown explicitly in Figure 9.19c are 
obviously available. The levels of the other nodes in subtrees rooted at the nodes 
q, %, @, fi, v4, and v) change by the same amounts as the levels of these 


corresponding nodes. 
As far as f(-) and v(-) are concerned, these stay the same for nodes that 
are not on the stem. For nodes i on the stem, we have f(i) = f(¥> ). Also, 2 (v) = 


1 + n(,) + n(v2), n(B) = 1+ n(fB,) + nv), n(Q) = 1 + n(Qy) + n(a@) + n(B), and 
n(q) equals the total number of nodes in 7, (the old n(v)). 


In conclusion, we mention one important expedient. Note that instead of 
the foregoing grafting procedure, we could alternatively have left J, rooted at v, 


rerooted 7, at p to get T,, and then grafted T, onto T, by connecting p and q. 


Then node v would have become the new root node. This process can be done 
exactly as previously. The tree that is rerooted at p or q is called the upper tree, 
while the other is called the Jower tree. The decision as to which tree to make 
the upper tree depends on the ease with which the rerooting of the “upper” tree 
can be effected. Since this effort is directly related to the length of the stem, the 
tree having the shorter stem q—v or p—r may be chosen as the upper tree. 


9.11 DEGENERACY, CYCLING, AND STALLING 


In the absence of degeneracy, the network simplex algorithm converges in a 
finite number of iterations as in Chapter 3. (By degeneracy, we mean a basic arc 
other than the root arc has a zero flow value; the root arc flow is zero for all 
feasible solutions) However, simple examples (see Exercise 9.50) have been 
constructed that show that the network simplex algorithm can cycle, that is, 
infinitely loop through a sequence of degenerate pivots. Hence, cycling preven- 
tion rules need to be employed. It turns out that a special lexicographic type of 
rule for the network simplex algorithm is not only easy to implement with no 
significant overheads, but is computationally beneficial for highly degenerate 
problems as, for example, the linear assignment problem. 

The prescribed cycling prevention rule is based on maintaining what are 
known as strongly feasible bases. A strongly feasible (basis) tree (SFT) for 
Problem (9.1) (including the root arc) is a feasible rooted spanning tree in which 
all degenerate arcs (if any) are pointing toward the root A directed arc (i, j) is 
said to be “pointing toward the root” if j = p(), that is, ifj lies on the chain from 
i to the root node r. It is instructive to identify the connection between strongly 


feasible and “‘lexicographically positive” bases of Chapter 4. The ith row of B! 
is given as the solution R, to the system R;B = e}. Observe that this is similar to 
the system “wB = cg.” Hence, R; may be found by giving the ith basic arc a 


cost of 1, the other arcs a cost of zero, and computing the dual variables. If the 
ith arc is a root arc, then R, has all ones (why?). If the ith arc is not a root arc, 


then from Section 9.5, the component of R; corresponding to node j is zero if 
the ith arc does not lie on the chain from j to the root. Otherwise, it is +1 if the 
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ith arc is pointing toward the root, and is —1 if the ith arc is pointing away from 
the root. In particular, for a basis corresponding to a strongly feasible tree, the 


row of B! corresponding to a degenerate arc has all components equal to 0 or 
1. In fact, the foregoing argument shows that a basis B has lexicographic 


positive rows in [B 'b, Bly if and only if it corresponds to a strongly feasible 


tree (why?). Hence, from Chapter 4 we are guaranteed finiteness if we maintain 
strongly feasible trees. 


Obtaining an Initial Strongly Feasible Basis 


If an all—artificial starting basis is used, then by simply defining the artificial arc 
connected to a transshipment node as being directed toward the (dummy (m + 
1)st) root node, we will have a strongly feasible starting basis. On the other 
hand, if some advanced starting basis is given, then consider the spanning graph 
formed by the arcs having positive flows and let some node be designated as the 
root node. This graph is a spanning forest. If it is connected, then it is a strongly 
feasible tree. Otherwise, one at a time, connect a component tree not containing 
the root node to the one containing the root node, using any arc in the problem 
that is pointing from a node in the former tree to one in the latter tree. Such an 
arc must of course be degenerate. Notice that if at any stage there are no arcs 
pointing from the set X = {i : node i is not as yet connected to the root} to the 
complement set X , then the problem is separable over the node sets X and X. 
This follows because we must have >);- yb; =Die¥ 0; =9, and the flow on any 


arc from a node in X to a node in X must be zero for all feasible solutions 
(why?). Hence, we may solve these separable problems independently. Note that 
we already have a strongly feasible tree for the node set X. Alternatively, we 
can connect any node in X to the root node via a degenerate artificial arc and 
continue this process. 


Maintaining Strongly Feasible Trees 


Given a strongly feasible basis tree at the beginning of an iteration, and given an 
entering arc (p, qg), the following simple rule for selecting a leaving variable arc 
(u, v) ensures that we maintain a strongly feasible tree. Trace the basis 
equivalent chain by tracing the unique chains from p and q to the root node r 
until they intersect at some common node k. Figure 9.20a depicts the situation 
generically. Note that & can coincide with p and/or r or with g and/or r. Define 
the cycle C(e) created as having the orientation of the flow along (p, qg) and 
commencing at node k, proceeding first along the k — p chain, then including 
arc (p, q), and finally proceeding along the g — k chain. Determine the exiting 
variable in C(e), breaking ties (if any) by selecting the Jast blocking arc in C(e). 
For example, in Figure 9.20b, when (4,7) enters, we have p = 4, q=7, andk=5, 
and of the blocking arcs (4, 3) and (6, 7), we select (6, 7) as the leaving variable 
arc. 

It is easy to see following Figure 9.20a that this rule maintains strongly 
feasible bases. Note that any degenerate arc not involved in the cycle C(e) is still 
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pointing toward the root after the pivot, irrespective of the choice for the leaving 
variable (why?). Observe that the pivot is degenerate if and only if there exists a 
degenerate arc on the chain k — p. In such a case, since the last degenerate arc 
on the chain k — p leaves the basis, the other degenerate arcs on k — p are still 
pointing toward the root, as are arcs (p, g) and any degenerate arcs that may 
have existed on the chain g — k. If the pivot is nondegenerate, then there are no 
degenerate arcs on the chain k — p. Moreover, any old degenerate arc on the 
chain g — k will now have a positive flow. Hence, we only need to examine the 
orientation of any new degenerate arcs created, which must correspond to alter- 
native blocking arcs. Because the last blocking arc in C(e) has been selected to 
leave the basis, the other blocking arcs are pointing toward the root after the 
pivot, and so we maintain a strongly feasible tree. 

In Chapter 4, it was proven that the foregoing algorithm will converge 
finitely. However, a more direct argument based on the network simplex 
algorithm is instructive. Note that it is sufficient to show that the algorithm 
cannot cycle (why?). Consider a degenerate pivot in which (p, qg) enters with 


Ong =Zpq ~€pq > 9. The leaving variable arc (u, v) must lie on the chain k — p. 


Hence, p is the leading node in the subtree 7, that is obtained when (u, v) is 


disconnected from the current tree and that does not contain the root node. 


Consequently, the dual values of the nodes in 7 fall by 6,,, while the dual 


values of the nodes in T — 7, remain the same. Therefore, in a sequence of degen- 


erate pivots, the sum of the dual variables, for example, provides a strictly 
(decreasing) monotone function that guarantees that a basis tree cannot repeat. 
This establishes the finite convergence of the network simplex algorithm. Note 


that as in Chapter 4, the vector (e 3B 'b,c zB!) is lexicographically decreasing 
over all iterations (why?). 


Gn arc 


LL 
arc 


: : Leaving ar 
orientation Ce) are 


eee arc 


Figure 9.20. Maintaining strongly feasible trees. (a) Generic situation. (b) 
An example. 


Minimal Cost Network Flows 491 


This cycling prevention rule is readily extended to the bounded variables 
network simplex algorithm. A strongly feasible basis tree may be defined here 
as one in which the basic arcs that are degenerate at their lower bounds are 
pointing toward the root and those basic and degenerate at their upper bounds 
are pointing away from the root. Then, the same leaving variable rule (which 
includes the case of the entering arc blocking itself) maintains strong feasibility 
and guarantees finite convergence. We leave the details to the reader in Exercise 
9.38. 


Stalling and its Prevention 


Although the foregoing technique guarantees finite convergence, it is still entirely 
possible that the algorithm may sfa// at an extreme point, that is, go through a 
number of consecutive degenerate pivots that, although finite, is exponential in 
m and n. For example, if b = 0 in Problem (9.1), then all possible bases 
represent the same extreme point, namely, the origin, and there may be an expo- 
nential number of such bases that one may enumerate before terminating with 
2, — Cy <0 for all nonbasic variables. In fact, there exist examples exhibiting 


this phenomenon (see the Notes and References section). In any case, it seems 
desirable to devise a rule that would guarantee that the number of consecutive 
degenerate pivots is bounded above by a polynomial in m and n. 

As seen in Chapter 4, a key to preventing stalling is to keep the length as 
well as the number of stages small. Recall that a stage is a sequence of degener- 
ate pivots in which no nonbasic variable remains enterable throughout and no 
strict subset of this sequence has the same property. The length of a stage may 
be controlled by adopting an appropriate “affirmative action” rule for selecting 
an entering variable. For example, the Least Recently Considered (LRC) vari- 
able choice rule discussed in Chapter 4 guarantees that the number of pivots in a 
stage is no more than n. It turns out that independent of the rule for choosing an 
entering variable, if we maintain strongly feasible basis trees as previously, then 
the number of stages in a sequence of consecutive degenerate pivots is no more 
than the number of degenerate basic variables at the current extreme point. 
Hence, the number of stages in a sequence of degenerate pivots is bounded 
above by m, so that the number of consecutive degenerate pivots using strongly 
feasible bases in conjunction with the LRC entering rule, for example, is 
bounded above by the polynomial mn. This underscores the relevance of main- 
taining strongly feasible bases. 

To establish the preceding bound on the number of stages, let 7, T),...,Ta4 


be the basis trees in a sequence of consecutive degenerate pivots such that either 
optimality is recognized at Tyy or the pivot executed on Tyy is nondegenerate. 


Notice that the nondegenerate basic arcs are common in all of the trees 7), 
7,24 (why?). Consider the spanning graph formed by these nondegenerate 
basic arcs. Let this graph have (tree) components G, Gp,...,Gp, where P < m. In 
each of the trees 7,...,T,4, these components Gj,....Gp are connected to each 


other via some P — | degenerate arcs. In particular, let T,, be as shown in Figure 
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9.21, with the root node re G,. Let LZ < P — 1 be the maximum number of 
degenerate arcs in the chain from any node to the root node in Tj,. Observe that 
P=7 and L =3 in Figure 9.21. We shall show that the number of stages in the 
sequence 7p,...,744 is no more than L, and hence no more than m — 1. 

Toward this end, let us make certain observations. First, since 7,...,Ty 


are strongly feasible and the pivots in this sequence are degenerate, the dual 
variable of each node is nonincreasing in this sequence and the sum is strictly 
decreasing. Second, since the leaving arc is degenerate in each pivot, the dual 
variables of the nodes in each subgraph G; either remain the same or all strictly 
decrease by the same amount for i = 1,...,P in each pivot. Third, denoting w as 
the dual values in Ty,, we have that w; = w,, for all i¢ G, in each of the trees 
7o,-...Jy, since the arcs in G; are basic in all these trees. Fourth, from the first 
point above, as long as the dual values of the nodes in any subgraph are not 
equal to their values in W, they are strictly greater than these values. 

By contradiction, suppose that the procedure goes through more than L 
stages. Consider a subgraph G, at the first “level” in Ty. By the “level” of a 


subgraph we will mean the number of degenerate arcs in a chain in Ty 


connecting any node in this subgraph to the root node. Suppose that for some 
tree T,, J € {0,...,44—- 1}, the dual values w; of nodes i € G, are strictly greater 


than w;. In particular, let (p, g) be the degenerate arc connecting G, to G, in Ty. 


=W,-W,-C,, >W, -W,-C 


Then, in the tree r; we have w p ~ Wy — lpg > Wy — Wy — lpg = 0 


pq —©pq 


Tree subgraphs having 
nondegenerate arcs 


Degenerate arcs 


Figure 9.21. Final tree 7), in a sequence of degenerate pivots. 
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(why?). As long as the dual values in G;, are not equal to those in W, arc (p, q) 
remains enterable and a stage persists. By the time the first stage is over within 
the sequence 7p,...,.7,,, we will have w; = w; for all ie G,. This equality holds 
in every tree after this stage. 

Inductively, assume that the procedure has gone through some @ > 1 
stages and the dual values of nodes in subgraphs at levels £ or lower are all at 
their corresponding values in W. Consider a subgraph G, at level (¢ + 1) and let 


(p, q) be the first degenerate arc encountered in the chain from any node in G;, 
to the root node. If w; > w, for ie G, in any tree following stage 2, then as 


before, in this tree we have Wp — Wg —Cpg =Wp —Wy —Cprg > Wy — Wy — Cog = 0 


so that arc (p, q) is enterable. As long as the dual values in G; are not fixed at 


their corresponding values in W, (p, qg) remains enterable and stage (¢ + 1) per- 
sists. When stage (¢ + 1) is completed within the sequence 7p,...,7,,, the dual 


values of nodes in subgraphs at level (2 + 1) are all at their values in W. Conse- 
quently, the number of stages in the sequence 7p,...,7,4, cannot be more than L. 
This establishes the required result. 


Another Useful Entering Variable Choice Rule that Prevents Stalling 


Besides the partial pricing rule for selecting an entering variable described in 
Chapter 3 and the LRC rule described in this chapter, there is another competi- 
tive entering variable selection rule that is well suited to how data are 
customarily stored. Suppose that the arcs in the problems are stored in a one— 
dimensional array with arcs having node 1 as their from—node appearing first, 
then arcs having node 2 as their from—node, and so on. Consider the following 
rule. Suppose that an arc having node / at its from—node has just entered. 
(Assume / = | to initialize.) Then the next entering variable is chosen as the best 
enterable arc with node again as its from—node, and if none exists, then the best 
enterable arc having node j + 1 as its from—node, and so on, considering arcs 
having nodes j + 2,..., m, 1, 2,..., and node j — | as their from—node, in this order. 
Then, we show that the maximum number of consecutive degenerate pivots 


using strongly feasible basis trees is no more than mL, which is less than m’, 


Note that the use of strongly feasible basis trees ensures that the number 
of stages in a sequence of degenerate pivots is no more than L, independent of 
the rule for choosing an entering variable. To show that the length of a stage is 
no more than m, consider a tree 7; in the sequence 79,...,7j4_; that leads to a 


degenerate pivot. Let w be the dual values in 7;. Suppose that arc (p, q) enters, 
say, where p € G,. Referring to Figure 9.21 (assuming that this represents tree 
si and not Tj), note that the pivot will be degenerate if and only if g does not 


lie in G, or in its “successors,” that is, if and only if the basis equivalent chain 
includes the degenerate arc (u, v), which is defined as the first degenerate arc 
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encountered by tracing from any node in G;, to the root node. Moreover, by the 
rule for breaking ties for the leaving variable (see Figure 9.20a), the arc (u, v) 
must be the leaving variable arc. Consequently, if (p, g) enters degenerately, 


then the dual values of nodes in G, and its successors reduce by Ong = Z pg —Cpg 


> 0, while those of the other nodes remain the same. Let these revised dual 
variables in T,,; be called w’. Because the set of nodes in G, or in its succes- 


sors have been unaffected by the pivot, for any node @ not in this set, we have 
U U 5 _ = = a, 2 _ = * 
Wy — Wp — Cpe = (Wy — Ong) — We — Cpe = (Wp — We — Cpe) — Spq SO since 6,4 was 


the most positive z —Cp, Value among all arcs of the type (p, f). If some arc 


pt 
having node p as its from—node is now enterable, the resulting pivot must be 
nondegenerate. Consequently, if the next pivot is degenerate, the from—node 
cannot be node p, which means that in a sequence of degenerate pivots the 
length of a stage is < m. This establishes the required bound. In Exercise 9.49 
we ask the reader to consider another variation of this rule. 


9.12 GENERALIZED NETWORK PROBLEMS 
Consider a linear programming problem of the form: 
Minimize ex 


subject to Ax 
b 


b 
0, 


IV tl 


where the matrix A is m x n of rank m and where each column a j of A has at 


most two nonzero entries. Then one may conceptualize the constraints Ax = b as 
a set of generalized nodal flow balance equations on a graph G(. # ,.°/) that is 


constructed as follows. Let. / be the set of m nodes {1,...,}, with node i being 
associated with the ith row of A. For each column a; of A having two nonzero 


entries, occurring, say, in rows p and q, construct an undirected link L; between 
nodes p and q. For each column a ; of A having a single nonzero entry, occurring, 
say, in row p, construct an undirected link L; incident at node p. Let . 7 denote 
the set of these links, and let x j denote the flow associated with link L - 


Additionally, for each node p, examine the links that are incident at p and 
associate with each such link L, at the point of incidence at p a multiplier equal 


to the coefficient in column a; and row p. Then the nodal balance equations 


assert that at each node p, the sum of the weighted flows on the incident links, 
where the weights equal the corresponding incident multipliers, must equal bp: 


Because of this interpretation of modified flows, this type of problem is called a 
generalized network or a network flow with gains problem. Problems of this 
type are used to model a variety of situations involving a gain or loss in flow or 
the transformation of one type of product to another in a network—based system. 
(See Exercises 9.54 — 9.55.) 
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As an illustration, consider the following example problem: 


Minimize 3x, + x, -— %3 + 2xq + X5 — 2% + x7 
subject to 2x, + 3x¢ = 6 
“xy + 3x, -— x7 = 0 
x3 O~«F 2x4 — X6 = 8 
3x3 + 2x5 + 3x7 = 8 
X47 XS = 
x x x x, x. x x, 20. 

1s 2? 3> 4> 5: 6° 7 


The corresponding graph G(. / ,.*“) is shown in Figure 9.22a along with the 


multipliers associated with the seven links. 
Consider the subgraph Ga(. / ,--%g) of G(. / ,.°/) shown in Figure 


9.22b, where the quantity 5, is shown against each node i, for i = 1,...,5. 


Suppose that the flows are permitted to be positive only on the links shown in 
Figure 9.22b, that is, the flows xg and x7 are restricted to zeros. Then at node 1, 


we have 2x; = 6 or x, = 3; at node 2, we must therefore have 3x7 — x, = 0 or xy 
= x,/3 = 1. Notice how the structure of the rooted tree is used to compute these 


values, as in the network simplex method. Similarly, treating the second 
component in Figure 9.22b as a tree for a fixed value of x3, we must have at 


node 3 that 2x, = 8 ~ x3 or x4 = 4 — (x3/2); we must have at node 4 that 2x, = 8 
— 3x3 or x5 = 4 — (3/2)x3, and we must have at node 5 that x4 — x; = 2. Substituting 
for x4 and xs in terms of x3 into this last equation gives x3 = 2, from which we 
obtain x4 = 3 and xs = 1. Hence, this solution, which is feasible and is uniquely 
obtained when x¢ and x7 are fixed at zero, is a basic feasible solution (why?). 


Notice how this directly generalizes the network simplex technique for solving 
the system Bx, = b. 


hay 


(a) (b) 


Figure 9.22. Example of a generalized network. (a) Network. (b) Basis. 
This example illustrates the following general characterization of a basis 
of A. Let B be any basis of A. Corresponding to B, construct a spanning sub- 
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graph Gp(. / ,--vg) of G(. / 4.7/7) as follows. Let - / ‘be the set of nodes 1.,..., 
m, and let .“vg be the set of links corresponding to the columns of B. Let 
Gail. / 5-7), 1 = 1,..., 7, be the r => 1 components of Gg. Then, as we show 
next, each component Gg3;, i = 1,...,r, is either a rooted tree or is a one—-tree. Such 
a graph Gp is called a pseudorooted spanning forest. Note that r = 2 in the 


example, and that one component is a rooted tree, while the other component is 
a one-tree. 


To establish the foregoing claim, let m; =| Jf i| be the number of nodes 
in component / for i = 1,..., 7. Note that since Gg; is connected, we have from 
Property 3 of tree graphs that |. 7g; |>(m;-1) for i = 1,..., . Suppose that for 
some i, we have | -“pi |= (m; —1), so that Gp; is a tree graph. Consider the m, 
rows of B corresponding to the nodes in . /;. Observe that these rows have 
nonzero entries only in the (m;-—1) columns corresponding to the links in . ““;, 


and so, these rows cannot be linearly independent, a contradiction to the fact 
that B is a basis. Consequently, we must have |. “7g; |= m; for i = 1,...,7. How- 


ever, L/_,m;=m and Y7_,|.7g;|=m, the total number of basic columns. 


Hence, we must have | “i |= m, for each i = 1.,,...,7. By our equivalent charac- 


terizations of tree graphs in Section 9.1, it then follows that each component, 
which is a connected graph on some m; nodes and having m, links, must either 


be a rooted tree graph or a one-tree graph. 
As a result of this characterization, any basis B of A can be arranged into 
r diagonal blocks Bj,...,B,, with B; representing the square (generalized) 


incidence matrix of component Gg; for i = 1,,...,.. Hence, the various operations 


associated with the simplex method, including the computation of dual 
variables, the generation of an updated column, and the pivot and update 
operations, can now be specialized to exploit this characterization and be 
performed essentially on the graph itself. 

For example, in order to solve the system wB = cz for the basis illus- 


trated in Figure 9.22b, we may compute (w,, w>) from the first component of the 
basis graph and compute (w3, w4, ws) from the second component by directly 


generalizing the network simplex technique using the appropriate link 
multipliers. This yields 3w, = 1 from the root arc incident at node 2 or w, = 1/3, 


and from link (1, 2) we get 2w, — w) = 3 or w, = 3/2 + w)/2 = 5/3. Similarly, 
from the second component, for a fixed value of w3, say, we obtain from link 
(3,4) that w; + 3w4 =—l, that is, wy = — 1/3 — w,/3, and from link (4,5) we 
obtain that 2w, — ws = 1, that is, ws = 2w4 — 1 = —S/3 — (2/3)w3, and finally, we 
obtain from link (3, 5) that 2w, + ws = 2. Substituting for ws in this last 
equation gives 2w3 — 5/3 — (2/3)w3; = 2 or w; = 11/4, which yields wg = —5S/4, 
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and ws = —7/2. Having now determined w = (5/3, 1/3, 11/4, -5/4, -7/2), we may 
price the nonbasic variables by computing z; —c; = wa;—c,. This gives 26 — cg 


= 3w W3 C6 = 5-11/4+2 = 17/4 and 27 —€7 = —Wy +3wy - c7 = -1/3 - 


15/4 — 1 = -61/12. Hence, xg is enterable. We may now generate the updated 
entering column y¢ by solving the system By, = ag in a manner similar to the 
solution of the system Bxpz = b, and then perform the pivot and updating 


operations. We leave the details of specifying the general rules for this 
procedure and the completion of the example to the reader in Exercise 9.48. 


EXERCISES 
[9.1] Solve the following network flow problem. 
bj 
5 6 é 
CX) 
wae 


| Oo 
i) 

(~) 
Ww 


[9.2] Suppose that the following figure represents a railroad network. The 
numbers beside each arc represent the time it takes to traverse the arc. Three 
locomotives are stationed at point 2 and one locomotive at point 1. Four 
locomotives are needed at point 6. Find the minimum total time solution to get 
the power required to point 6. 


[9.3] Put the following problem in the standard form (Problem (9.1)). Construct 
an all artificial initial basis and optimize using the two-phase or the big—-M 
method. 
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Minimize 4x,> +3x13 +2Xx53 ~2Xx49 +634 43X35 +3.x64 + X65 —4x75 +2x76 


subject to x9 + x43 < 
—%2 + X93 — X42 =-2 
— X43 — X93 + X34+ X35 = 3 
+ X42 — %34 — X64 <6 
— %35 X65 — X75 at 
+ X64 + X65 _ x76 = 2 


X22 X13, X03, X42, X34, 35> X64 X65, X75, X62 0. 


[9.4] Solve the following flow problem. (Note: +b; #0.) Does it have alterna- 
tive optimal solutions? 


[9.5] Consider the following network flow problem. 


In addition: Node 1 can produce an unlimited supply of units at a cost of two per 
unit; Node 2 can produce up to three units at a cost of four per unit; Node 3 
needs three units; and Node 4 needs five units. Find an optimal production and 


shipping policy. 


[9.6] Solve the following transportation problem by the network simplex 
method of this chapter: 
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[9.7] If it is known in advance that a certain arc will be carrying positive flow in 
any optimal solution to an uncapacitated network flow problem, what 
simplifications can be made in the solution method? 


[9.8] Show that the two root arcs and the chain between them form a dependent 


. = 


[9.9] Provide a complete detailed proof for the equivalence of the six 
characterizations of a tree given in Section 9.2. 


[9.10] Solve the following network flow problem using x3, x34, and x42 as 
part of a starting basis. 


b=3 by =5 


[9.11] It is often necessary (particularly in the dual simplex algorithm and in 
integer programs) to generate a row of the simplex tableau. Indicate how this 
can easily be done for a network flow problem. Illustrate by generating the row 
associated with x,;3 in the starting simplex tableau for Exercise 9.10. [Hint: 
Remove some arc (basic in a given row) of the basis tree, thereby separating the 
nodes into two subsets. Then consider the set of nonbasic arcs going between 
the two node sets—those in the same direction as the given basic arc and those 
in the opposite direction.] 


[9.12] Indicate how we can generate a row or a column of B™! associated with 


a network flow problem. Illustrate by generating the row of B! associated with 
X13 in the initial basis for Exercise 9.10. 


[9.13] Apply the two-phase method and the big—M method to the network of 
Exercise 9.10 to find an optimal solution. 


[9.14] Draw the underlying graph for the transformed problem of Example 9.2. 
Optimize using the big—M method. Start with an all artificial variable basis. 


[9.15] Prove or give a counterexample: In order for a variable to be basic in a 
given row of the linear program for a network flow problem, the variable must 
have a nonzero entry (that is, +1) in that row of the original constraint matrix; 
that is, the arc must be incident with the associated node. 


[9.16] According to the rule of Section 9.5, show that the root is the last basic 
variable to be assigned a value. 
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[9.17] In the network flow problem of Section 9.1, suppose that we locate a 
path from a supply point to a demand point. Putting as much flow as possible on 
the path, decreasing the corresponding supply and demand, suppose that we 
repeat this process until all supplies are allocated (and demands satisfied). 


a. Will the feasible solution obtained be basic? 
b. Ifnot, how can the solution obtained be made basic? 


[9.18] Solve the following network flow problem. (Note: For this choice of b;— 
values, we will have found a shortest path from node | to node 4.) 


[9.19] Consider the following network. Node | has five units available. Node 3 
has two units available. Node 2 needs four units. Node 4 needs one unit. 


a. Set up the linear program for this problem. 
b. Solve the problem by the network simplex method. Is the optimal 
solution obtained degenerate? 


[9.20] Starting with x,2, x24, and x3, as part of a basis where x,4 is nonbasic 


at its upper bound and where all other x, variables are nonbasic at their lower 


bounds, solve the following network flow problem: 
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[9.21] Apply the two-phase method and the big—/ method to Exercise 9.20. 
[9.22] Solve the following network flow problem: 


[9.24] Indicate how we can handle lower and upper capacity constraints on flow 
through a node i. (Hint: Consider splitting node i into two nodes.) 


aa y ~ (é, u) 
POX 


[9.25] Consider the following network of cities. Each city must be visited 
exactly once by some salesman traveling in a nontrivial circuit. Use network 
flows to indicate the number of salesmen necessary and their routes to minimize 
the total distance traveled. Assume that an unlimited number of salesmen are 
available at no cost to be positioned wherever needed. (Hint: See Exercise 9.24.) 
Can we impose the additional constraint that only one salesman be used? (This 
would be the classic traveling salesman problem.) 
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[9.26] Show that a network flow problem having zero lower bounds and finite 
upper bounds can be converted to one having zero lower bounds and no 
(infinite) upper bounds. [Hint: Consider splitting arc (i, /) as shown. ] 


PO! GX oe G) (0,00,0) a 


b; b; b; -uy by +uy 


Derive this transformation algebraically by performing appropriate row opera- 
tions after converting the upper bounding constraints to equalities. Illustrate by 
converting the following network to one without upper bounds: 


[9.27] Indicate how any finite—valued, uncapacitated, minimal—cost flow prob- 
lem of Section 9.1 can be transformed into a transportation problem. Illustrate 
by the network shown. (Hint: Locate the minimal cost path from each supply 
point to each demand point.) How would you achieve this transformation for 
capacitated problems? (Hint: See Exercise 9.26.) 


[9.28] Consider the following network for a minimum-—cost network flow 
problem with the net supply values (6;) shown on the nodes and the arc costs cj 


shown on the arcs: 
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a. Construct a rooted spanning tree with Node 2 as the root node and 
the arcs (1, 2), (2, 3), (2, 5), (3, 6), and (4, 5) along with the root arc 
as basic. Determine the associated primal and dual solutions for this 
basis. 

b. Show that this basis is dual feasible. On the graph, perform a dual 
simplex iteration in which the dual variable complementary to x5 
enters the (dual) basis. Show all details. 

[9.29] Consider a feasible minimum-cost network flow programming problem 
(9.1) being solved using an all-artificial start basis along with the big-M 
method. In terms of the cost on chains from nodes to the root node, prescribe a 
value of M that is sufficiently large. (Hint: Consider any optimal basis to the 
original problem and determine how big M should be so that the artificial 


nonbasic arcs have 2 —¢y <0.) 


[9.30] Indicate how the lexicographic simplex method may be applied to the 
network flow problem. Identify each step with that using strongly feasible basis 
trees. 

[9.31] Develop in detail two algorithms for solving network flow problems that 
are respectively based on the dual simplex and the primal—dual methods. 

[9.32] Suppose that a minimum-—cost network flow program contains an 
undirected arc (i, j) with a cost per unit flow of c;. Show how the corresponding 


variable x;; appears in the mathematical formulation of the problem before using 
any transformations on this variable. 


[9.33] How can we handle undirected arcs when £ = 0 and c > 0 in a network 
flow problem? (Hint: Consider replacing a single undirected arc by two 
oppositely directed arcs.) What happens when cj <0 or when ¢ i 7 0 for some 


arc(s) (i, /)? Relate the transformation used to solve the problem of Exercise 


9.32. 
OO OF50 


(u, ¢) 


[9.34] Show how a network flow problem with £2 = 0, for which the cost 
function associated with each arc is piecewise linear and convex, can be solved 
by the methods of this chapter. (Hint: Consider adding parallel arcs—one for 
each segment—with proper costs and upper bounds.) Can the methods of this 
chapter be also used for network problems having piecewise linear and concave 
functions? Explain. 


[9.35] How can we transform a network flow problem with some zy 70 into 


one with all 2=0? Illustrate by the following network: 
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[9.36] Consider the following network flow problem with the b;—values shown 


on the nodes and with the triple (2 cj) shown on the arcs. Convert this 


ee) 
y > yy? 
problem to an equivalent network flow problem having all lower bounds equal 
to zero. 


[9.37] Prove that when we discard a redundant constraint, a network flow basis 
is characterized by a rooted spanning forest (a collection of trees, each having 
exactly one root, which spans the node set). Using this characterization, develop 
the representation of a nonbasic column in terms of the basic columns. Utilize 
your results to describe a complete simplex method for network flow problems 
without any redundant constraint. Apply the method to the network in Exercise 
9.10 with the last constraint deleted. 


[9.38] Using the definition given in Section 9.11 of a strongly feasible basis tree 
for bounded variables network flow problems, show how the rule given there 
maintains strong feasibility and prevents cycling. (Hint: See Exercise 9.26!) 

[9.39] A company has requirements for a certain type of machine during the 
next N months of D; per month, i = 1, 2,..., N. These requirements must be met, 
although excesses are permitted to any desirable extent. No machines are avail- 
able at the beginning of month 1. At the beginning of each of the N months, the 
company may purchase machines. The machines are delivered immediately; that 
is, there is no lead time. The company may purchase machines that last one 
month, two months, and so on up to M months. The number of months a 
machine is usable is called its service life. The cost of a machine depends on the 
month in which it is bought and on its service life. In particular, a machine 
bought in the ith month with service life of k months costs c,, dollars. Naturally, a 


machine that lasts longer costs more, so that cj, < cj, for p <s. 


a. Formulate a mathematical statement for this problem, assuming that 
the objective is to minimize the sum of the monthly costs. Let x, be 
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the number of machines bought in month i with service life of k 
months. 

b. Derive an equivalent formulation for the general problem as a net- 
work flow problem. Use N = 4 and M = 3 to illustrate, summarizing 
the resultant formulation on a network. (Hint: Consider elementary 
row operations to obtain a | and —1 in every column.) 


[9.40] The following network represents an electrical power distribution net- 
work connecting power generating points with power consuming points. The 
arcs are undirected; that is, power may flow in either direction. Points 1, 4, 7, 
and 8 are generation points with generating capacities and unit costs given by 
the following table. 
Generating Point 
] 4 7 8 

Capacity (thousands of 100 60 80 150 

kilowatt hours) 

Unit Cost ($/1000 kilowatt 50.0 63.5 75.0 84.5 

hours) 
Points 2, 5, 6, and 9 are consuming points with demands of 35,000 kwh, 50,000 
KWH, 60,000 kwh, and 40,000 kwh, respectively. There is no upper bound on 


the transmission line capacity. The unit cost of transmission on each line 
segment is $25.0 per 1000 kwh. 


a. Set up the power distribution problem as a network flow problem. 
b. Solve the resulting problem. 


[9.41] Consider the following production-transportation—inventory problem. A 
firm produces a single product at two locations in each of two time periods. The 
unit costs and production limits vary by time period and are given in the follow- 
ing table: 


Production Time Period 
Location ] 2 
] $30/6 $50/2 Unit cost/ 
2 $35/10 $57/9 Production limit 


The product will be shipped (instantaneously) to each of two locations to satisfy 
specified demands over the two periods. These demands are as follows: 


Consumer Time Period 
Location ] 2 
] 3 ] 


2 P) 4 
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The unit shipping cost varies over time and is given by the following: 


Period 1; Consumer 
Location: 


Period 2; Consumer 


Production Sige 
Location: 


Location 


Production 
Location 


2 
$65 
$75, 


2 
$90 
$100 


Finally, the product may be held in inventory at the production and consumer 
locations to satisfy later period needs. The relevant data are given below. 


Consumer Location 
1 2 


$3/1 $5/3 


Production Location 
1 


$1/2 


Unit cost/ Inventory 
limit 


Set up a network flow problem that can be used to solve the problem of 
minimizing the total cost to satisfy the demand over the two periods. (Hint: 
Create a separate node that represents each location at each time period. 
Shipping arcs connect nodes for the same time period; inventory arcs connect 
nodes in different time periods.) 


[9.42] Formulate Exercise 1.16 as a network flow problem. Solve this problem 
by the network simplex method. 


[9.43] Thirty million barrels of oil must be transported from Dhahran in Saudi 
Arabia to the ports of Rotterdam, Marseilles, and Naples in Europe. The 
demands of these ports are, respectively, 8, 18, and 4 million barrels. The 
following three alternative routes are possible (see accompanying map). 


a. From Dhahran, around Africa (through the Cape of Good Hope) to 
Rotterdam, Marseilles, and Naples. The average transportation and han- 
dling cost per barrel is $2.20, $2.40, and $2.45, respectively. 

b. From Dhahran to the city of Suez, and then through the Suez Canal to Port 
Said. From Port Said the oil is shipped to Rotterdam, Marseilles, and 
Naples. The average transportation and handling cost from Dhahran to the 
city of Suez is $0.35 and the additional unit cost of transporting through the 
canal is $0.25. Finally, the unit transportation costs from Port Said to 
Rotterdam, Marseilles, and Naples are, respectively, $0.30, $0.23, and 
$0.20. 

c. From Dhahran to the city of Suez, and then through the proposed pipeline 
system from Suez to Alexandria. The average transportation cost per 
barrel through the pipeline system is $0.17, and the unit transportation 
costs from Alexandria to Rotterdam, Marseilles, and Naples are $0.28, 
$0.22, and $0.20. 

Furthermore, 35 percent of oil in Dhahran is transported by large tankers that 
cannot pass through the Suez Canal. Also, the pipeline system from Suez to 

Alexandria has a capacity of 10 million barrels of oil. 


a. Formulate the problem as a general network flow problem. 
b. Use the procedures of this chapter to find an optimal shipping pattern. 
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[9.44] a. State the dual problem for the lower—upper bounded network flow 


problem in Section 9.8. 
b. Give the values of all of the optimal dual variables for Figure 9.17 at 


iteration 2. 
c. Verify optimality by computing the dual objective function. 


[9.45] Consider the following network: 
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Consider a basis given by x21, X23, X34, X52 and the artificial (root arc) variable 
Xq With x;4 nonbasic at its upper bound and all other variables nonbasic at their 
lower bounds, that is, zero. 


a. Find the basic solution. Is it feasible? 

b. Give the basis in lower triangular form. 

c. Give the simplex tableau associated with the specified basic solution. 

d. State the dual program. 

e. Find the complementary dual solution. Is it feasible? 

f. Regardless of costs, perform one pivot on the network to bring x,4 into 
the basis. 

g. Is the new basis optimal? 


[9.46] Show that the coefficient matrix for the lower-upper bounded network 
flow problem is totally unimodular. This matrix is of the form: 


x Sy 83 RHS 
A 0 0 b 
I -I 0 £ 
I 0 I u 


[9.47] a. Provide the values of p(i), €(1), (i), tr(), Aid), n(@) for i = r, 1,...,27 
corresponding to the tree in Figure 9.18. 

b. If this was an advanced starting basis, show how you would construct 
these lists (without using a graphical display) and how you would com- 
pute the flow and dual values using these lists. 

c. For the pivot shown in Figure 9.18, provide a precise account of how 
these lists and quantities are updated using 7, as the lower tree. 


d. Repeat Part (c) using 7, as the lower tree. 


(9.48] Consider the generalized network problem introduced in Section 9.12. 
Given the characterization of a basis as a pseudorooted spanning forest, develop 
specialized rules for computing the complementary dual basic solution, generat- 
ing the updated column of an entering variable, and for performing the pivot 
operation and updating the basis along with the associated primal and dual 
solutions. Illustrate using the example of Section 9.12. 


[9.49] Consider a minimum-—cost network flow problem in which the nodes are 
numbered 1, 2,...,2 and the arcs are stored in the same order as those having 
node | as their from—node, then those having Node 2 as their from—node, and so 
on. Suppose that the following rule for selecting an entering variable is adopted. 
If an arc having node j as its from—node was last selected, pick the next arc as 
the best enterable arc having the earliest node in the list j + 1,...,7, 1, 2,...,/ as its 
from—node. Using this rule in conjunction with strongly feasible trees, show that 
the maximum number of consecutive degenerate pivots is bounded above by 
mL, where L is the largest number of degenerate arcs in any chain from a node 
to the root node in the final tree T), of the degenerate sequence. (Hint: Follow 
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the original stalling prevention proof of Section 9.11. Note that it is no longer 
necessarily true that the stage length is less than or equal to m in this case.) 
[9.50] Consider a network flow problem with three vertices that each have b; = 
0 and with twelve arcs given as follows. Between each pair of nodes i and j, 
there are two arcs from i to 7 having costs +1 and —1, respectively, and there are 
also two reverse arcs from j to i having costs +1 and —1, respectively. Consider a 
starting basis in which the arc (1, 2) with cost 1 and the arc (2, 3) with cost —1 
are basic. By always keeping one arc with a cost 1 and one arc with a cost —1 
basic, and entering appropriate arcs in the cyclic order between | and 3, then 
between 2 and 3, and next between | and 2, starting with the entering arc from 1 
to 3 with a cost —1, construct a sequence of bases that exhibits the phenomenon 
of cycling. (Hint: Try to construct a cycle of length 12, that is, trees 7, 7],...711, 
T,2, where 7, = To, in which for each i = 0, 1,...,5, the tree 7}, is identical to 7; 
except that the two basic arcs of the same cost have reverse orientations. ) 

[9.51] Consider a directed graph G(. / ,.~”) and suppose that a two—-commodity 
minimum-—cost network flow problem is defined on this digraph, where x and y 
denote the flow variables for the two respective commodities. 


Minimize ex+dy 


subject to LY xy- Ly =; fori=l,....m 
PO)E7 kik, i)e-7 
cL vy- Lg = OQ, fori=l,...,m 
FU J)E 7 kk, i)e.-7 
AXx<y <x 
x,y 20, 


where 0 < @ < @, are given constants. 


a. Add nonnegative slack variable vectors s, and s, to write Ax <y <@)x 
as &x-y+s, = 0 and y—@x+s, = 0. Show that the nonnegativity 


constraints on x and y are implied by the other constraints. 

b. Treating x and y as unrestricted above, solve for them in terms of the 
slack variables and eliminate them from the problem. Demonstrate that 
the resulting problem is a network flow problem separable in the vari- 
ables s, and s>. 


Cc: Comment on the algorithmic implications of this strategy. 
[9.52] Develop a method to solve a linear program of the form: 
Minimize ¢x+¢,41%41 


subjectto Ax+a,,1%,4; =b 
x2 0,x,4) 20, 


where A is a node—arc incidence matrix. Apply the method to the following 
problem: 
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Minimize 5x + 3X, + 4x3 + 3x4 + 5X5 + X6 
subjectto x + Xx» + X% = 2 
xy + x3 + X4 = 2X6 = 3 
-— X% - % + X5 + 2x, = -l 
-— Xp -— Xs -— X = -4 
x; X, X3, X45 X55 X% 2 O 


[9.53] Can the results of Exercise 9.52 be generalized to the case where the 
constraint matrix is of the form (A, D), where A is a node~arc incidence matrix 
and D is any arbitrary matrix? Apply the method to the previous problem with 


the additional column x7 having a7 = (1, -6, 2,0 y and cz =-4. 


[9.54] Consider the investment problem presented in Exercise 1.1. Construct a 
generalized network to represent this flow with gains problem. Solve this 
problem using the algorithm discussed in Section 9.12. 

[9.54] In the copper market, copper ore is mined at different mines i = 1,...,M/, 
having respective supplies s; tons, i= 1,...,/, and shipped to various smelters j 
= ],...,8, where it is processed to produce blister copper, which is then sent to 
different refineries, k = 1,...,K, which produce high purity copper. This final 
product is used to satisfy demands d,, 4 = 1,...,L, occurring in some L end-use 
markets. There also exist predominantly copper-based scrap metal markets r = 
1,...,R, having respective supplies o,. tons, r = 1,...,R, which feed back copper 
into the system by shipping this copper—based scrap metal to the refineries for 
purification. Assume that a ton of ore sent from mine i to smelter 7 will produce 
a; €(0,1) tons of blister copper at smelter j at a total mining, transportation, 


and smelting cost of Cy per ton; a ton of blister copper sent from smelter j to 


refinery k will produce Bj, €(0,1) tons of refined copper at a total 


transportation and refining cost of Cir per ton; a ton of scrap metal sent from 
scrap metal market r to refinery k will produce y,, €(0,1) tons of refined 
copper at a total transportation and refining cost of cy per ton; and that the 


distribution cost from each refinery k to end-use market £ is chy per ton of 
refined copper. For this simplified single period description of the copper 
market, construct a generalized network of the form depicted in Figure 9.22a to 
find a least cost copper processing and distribution solution, displaying the 5,;- 
values at the nodes, and the A—matrix elements and cost coefficients on the 
edges. Define your decision variables associated with the different types of 
edges. Discuss how you might extend this model to a multi-period situation. 
(Soyster and Sherali (1981) describe a more sophisticated multi-period model 
for the U.S. copper market.) 


[9.55] Consider the linear program LP to minimize cx subject to Ax = b, x > 0, 
where A is an m x # matrix of rank m <n. Consider any column a j of A 
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associated with the variable x;. For example, let aj; = (3, 2, -4, 6, 0,.. .,0)', and 


suppose that we write 


0 
3 0 
4 
[a; |x; =| 0 |x; +| 6 |x;2, where we additionally impose xj) = x;2(= x;). 
; 0 
0 . 
0 
(This technique is called variable splitting based on its operation, and the 
relationship of the type xj; =x;2 1s referred to as an equal-flow constraint.) 


Show how the general linear program LP can thus be converted to a generalized 
network flow problem with equal—flow side—constraints. Referring to the 
Lagrangian relaxation decomposition strategy in Chapter 7, discuss how you 
might exploit this induced structure to solve LP. 


NOTES AND REFERENCES 


1. 


The simplex method for general network flow problems is a natural exten- 
sion of the work on the transportation problem and the work of 
Koopmans [1949] relating linear programming bases for transportation 
problems and trees in a graph. On this subject, also see Ahuja et al. [1993] 
and Bertsekas [1991, 1998]. 

Computational experience with the network simplex algorithm, as 
reported by Glover et al. [1974a, b], Langley and Kennington [1973], Ali 
et al. [1978], and others, indicates that this algorithm compares favorably 
with several other procedures for solving network flow problems. 
However, recent work by Bertsekas [1985] and Bertsekas and Tseng 
[1988a, b; 1994] has resulted in a primal—dual type of algorithm (see 
Chapter 11) that appears to dominate the primal simplex algorithm. The 
codes RELAX and RELAXT based on this algorithm solve the standard 
NETGEN benchmark test problems (Klingman et al. [1974]) two—four 
times faster than the primal simplex code RNET written at Rutgers 
University. All commercial LP solvers have a specialized option to solve 
network structured problems (e.g., the popular software CPLEX has the 
NETOPT option based on the network simplex algorithm). For an 
application of interior point methods to network flow problems, see 
Mehrotra and Wang [1996]. 

Zadeh [1973] has exhibited the worst-case exponential behavior of the 
primal simplex and the primal—dual network algorithm among others, 
using a class of modified transportation problems. Edmonds and Karp 
[1972] were the first to propose a polynomial-time algorithm for network 
flow problems. Their procedure is based on a scaling technique that 
solves a sequence of network flow problems with modified (scaled) 
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right—hand-—side values, where the solution to each problem is obtained 
from the previous one via at most m shortest path problems, and where 
the first problem is a shortest path problem. Tardos [1985] and Orlin 
[1984] showed how this algorithm can be modified to derive strongly 
polynomial dual simplex types of algorithms for minimum cost network 
flow programming problems. A first polynomial-time primal simplex 
type of network flow algorithm is presented in Orlin [1997]. (Also, see 
Sokkalingham et al. [2000].) 

The discussion of list structures in Section 9.10 is based on the 
enhancements to the augmented threaded index (ATI) method of Glover 
et al. [1974a] as discussed in Barr et al. [1979]. (See Barr et al. [1979] fora 
discussion on rerooting 7,, at q based on reverse threads as an alternative 


to the technique presented here.) Glover and Klingman [1982] present 
some computational comparison studies. For an alternative to perform 
tree—based operations using a data structure called dynamic trees, see 
Sleator and Tarjan [1983, 1985]. 

The strongly feasible bases of Section 9.11 were introduced by 
Cunningham [1976]. Their equivalence with lexicographic positivity was 
exhibited by Orlin [1985]. The instance of cycling in Exercise 9.50 is 
from Cunningham [1979]. Cunningham [1979] also exhibits the occur- 
rence of stalling and discusses its prevention. Orlin [1985] has shown that 
using Dantzig’s entering criterion along with the lexicographic cycling 
prevention rule, the maximum number of consecutive degenerate (primal 
simplex) pivots for the minimum cost network flow problem is bounded 
by O(n? n log m). 

A general technique for converting linear programs to network flow 
problems wherever possible via elementary row operations and variable scal- 
ing (i.e., finding hidden networks) is presented in Bixby and Cunningham 
[1980] (see Exercise 9.39). See also Klingman [1977]. Cunningham 
[1983] shows an instance in which the elimination of unrestricted 
variables can help in such a conversion process. In cases where a com- 
plete conversion is not possible, methods for exploiting network substruc- 
tures are discussed in Glover et al. [1978], Glover and Klingman [1981], 
and Frendewey and Glover [1981]. Also see Patty [1987] for a survey. 
Section 9.12 (also see Exercise 9.48) discusses the extension of the 
models of this chapter to the flow with gains or the generalized network 
models. Jewell [1962] first solved this problem by a primal—dual method. 
Ellis Johnson [1965], Langley [1973], and others have since treated the 
problem via the simplex method. For further reading on this subject, see 
McBride [1981], Elam et al. [1979], Glover et al. [1974b], Kennington and 
Helgason [1980], and Bertsekas and Tseng [1988a, b; 1994]. Goldfarb 
and Lin [2002] discuss interior point methods for this class of problems. 


TEN: THE TRANSPORTATION 
AND ASSIGNMENT 
PROBLEMS 


Transportation and assignment problems are important network structured linear 
programming problems that arise in several contexts and that have deservedly 
received a great deal of attention in the literature. Although these problems are 
special cases of network flow problems, we have seen in the previous chapter 
that any finite-valued, capacitated or uncapacitated, minimum-—cost network 
flow problem can be transformed into an equivalent (uncapacitated) transporta- 
tion problem (see Exercise 9.27). In fact, although the assignment problem is 
itself a special case of the transportation problem, any transportation problem, 
and hence any (capacitated) minimum-cost network flow problem, can be 
equivalently transformed into an assignment problem, as shown in Section 10.7. 
However, this is computationally inadvisable. Nonetheless, this feature is one 
reason for the interest generated by this pair of problems. 

Algorithmically, the uncapacitated or the capacitated transportation 
problem may be solved using the techniques of the previous chapter. Some 
specialization in implementation is possible, but for the most part, the basic 
technique remains the same. For the sake of exposition, we will present some of 
these specializations in the context of the transportation tableau, which is 
customarily used to pedagogically present the network simplex algorithm for 
transportation problems. Although we will rely on the previous chapter for justi- 
fications and proofs, and we will exhibit the relationships of the method presented 
here with the general techniques of the foregoing chapter, the discussion in the 
present chapter is more or less self-contained. 

On the other hand, because of its very special structure, a host of 
specialized algorithms have been proposed for the assignment problem. The 
well-known and instructive Hungarian algorithm, which led to the development 
of a general primal—dual algorithm for linear programming problems, is pre- 
sented here. We also comment on a specialization of the strongly feasible 
network simplex algorithm for assignment problems, known as the alternating 
basis algorithm, which yields a competitive solution procedure. Another algo- 
rithm that runs up to seven times faster than the latter algorithm is based on a 
successive shortest-path procedure. We present this method here, but postpone 
the actual solution of the shortest-path subproblems to Chapter 12. 


10.1 DEFINITION OF THE TRANSPORTATION PROBLEM 


Consider m origin points, where origin i has a supply of s; units of a particular 
item (commodity). In addition, suppose that there are n destination points, where 
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destination j requires d; units of the commodity. We assume that s;, d; > 0. 


Associated with each link (i, /), from origin i to destination /, there is a unit cost 
cj for transportation. The problem is to determine a feasible “shipping pattern” 


from origins to destinations that minimizes the total transportation cost. This 
problem is known as the Hitchcock or the transportation problem. 
Let x, be the number of units shipped along link (i, /) from origin i to 


destination j. Furthermore, assume that the problem is balanced, that is, the total 
supply equals the total demand. Hence, 


If the total supply exceeds the total demand, then a dummy destination can be 
created having demand d,,,; = 2j5;-Ljd;, and ¢;,4; = 0 for i = 1,..., m. 


Assuming that the total supply equals the total demand, the linear programming 
model for the transportation problem becomes as follows. 


Minimize 
Cypyp tees Heyy + CqyX%qy Fo Hla pXan Fo HC Xm + tn mn 
subject to 
Aptos + xy = 3 
x2] + + X2n = 59 
Xm) + +Xun = Sim 
x11 3 | see A Xml = d, 
Xin + Xn +Xinn = dy, 
HX Jovee Xin» XQ] 5 v0%9 X2no vee Xml> see Xmn > 0. 


The transportation problem is graphically illustrated in Figure 10.1. 
The underlying graph, which is comprised of the origin and destination 
nodes O;, i= 1,..., m and Dj»j = 1,..., 7, respectively, and the connecting links or 


arcs, is said to be bipartite. That is, the nodes are partitioned into two sets such 
that all the arcs in the network are directed from a node in the first set to a node 
in the second set. It is a complete bipartite graph in the sense that all such 
possible arcs are present. Here, we assume that if any connection between an 
origin-destination pair (i, j) is prohibited, then the corresponding cost 
coefficient cj is large enough so that the variable xy is essentially an artificial 
variable in the problem. The problem also may be represented by a 
transportation tableau in which the rows 1,...,7 represent the origin nodes, the 
columns j = 1,..., m represent the destination nodes, and the cell in row i and 
column j represents the flow variable x,,. The corresponding cost coefficient cy 


is often displayed as shown in cell (i, /). 
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Figure 10.1. Illustration of the graph of a transportation problem. 


Destinations 


1 St 
2 Sy 
Origins : Supplies 
5; 
m Sim 


Demands 


We can cast the transportation problem in matrix form if we let 
= t 
X= (5% ]0 5005 Xp 9X2] 5209 X29 99 Xm ) 
© = (Cy 5 C2 5-065 C]y 9 C2] 9229 C29 9-9 Cn ) 
t 
b= (51559 5.04555 ,—-2,...5 Ay) 


A= (By 158] 9 50065] 9 BD] 9-29 B29 9-0 Ary ) 
where 


ay =; ~ ems 7 


and e; and e,,, ; are unit vectors in R”*”, with ones in the ith and (m + /)th 


positions, respectively. The reader should note that we have multiplied the 
demand constraints through by —1 in order to be consistent with Chapter 9. An 
identical derivation without this operation is readily evident. With these 
definitions the problem takes the following form: 


Minimize ex 
subject to Ax 
x 


IV Il 
i) 
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The A matrix, which is the node—arc incidence matrix defined in Chapter 9 and 
which is of dimension (m + n) x mn has the following special form: 


mn columns 
1 O.: 0 
0 1. 0 
A=| : : : |m+n rows 
0 O.- 1 
ee | 


where 1 is an n—-row vector of all ones and I is an 7 x n identity matrix. It is the 
A matrix that gives the transportation problem its special structure. 

As an example of a transportation problem, consider a 2-origin, 3- 
destination transportation problem with data as indicated below. 


Destination 
l 2 3 S; 
! aes] 3 
Origin 2 20 
d; 15 10 25 
For this problem, 
1 1 1 0 0 0 
0 0 0 1 1 1 
A=/-l1 0 0-1 0 0 
0-1 0 0 -1 0O 
0 0-1 0 0 -1 


Feasibility of the Transportation Problem 


Under the assumption that total supply equals total demand, the transportation 
problem always has a feasible solution. For example, it is easy to show that 


2M) 
xy = 7 


where d= ));5; =X j d p is a feasible solution. Note that for each feasible vector 


‘ i=1,...,m, J=1,....7, 


x, every component x;; is bounded as follows: 


O<x,; < minimum{s;,d;}. 


We know that a bounded linear program having a feasible solution has an 


optimal solution. Thus, we now need to describe how to find an optimal 
solution. 


10.2 PROPERTIES OF THE A MATRIX 


We shall examine some of the properties of the A matrix that give the 
transportation problem its special structure. As we shall see, these properties 
permit a simple and efficient application of the simplex method for transporta- 
tion problems. 
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Rank of the A Matrix 


Clearly, rank (A) 4 m + n, since the sum of the rows equals zero. In fact, as seen 
in Chapter 9, the rank of A is m+ -— 1. Hence, we are left with two choices for 
a basis~—-we can either delete the last row, or any row, leaving m + n— 1 linearly 
independent constraints for which a basis exists, or we can add an artificial 
variable in one constraint. When applying the simplex method, we shall select 
the latter approach and augment A with a new artificial or root arc variable x, 


having a column e,,,,,. Henceforth, we denote A° =(A, ere 


Total Unimodularity of the A° Matrix 


The single most important property that the transportation matrix possesses is 


the total unimodularity property. In Chapter 9, we showed that the A°® matrix is 
totally unimodular, that is, the determinant of every square submatrix formed 


from it has value —1, 0, or +1. This means that every basis B of A°, and hence 


its inverse, has determinant +1, and that the elements of B™! are all +1 or 0 
(why?). Using Cramer’s Rule (see Section 2.2), we conclude that the updated 
simplex column y,; of the variable x, in a canonical representation with respect 


to the basis B, which is given as a solution to the linear system By; =ajj, is 
comprised of elements that are +1 or 0. This means that any vector aj can be 


obtained by the simple addition and subtraction of basic vectors. It is instructive 
to see how we can construct such a representation on the transportation tableau. 
In particular, in the representation of the nonbasic vector ay = €; —€m4 ; 


in terms of basic vectors, there must be a basic vector of the form ay = 


e; —e,,4, with a coefficient of +1. Then there must exist a basic vector of the 


i 
form ay, =e, —e,,,, With a coefficient of -1 in the representation. This process 


continues until finally there must exist a vector of the form a,; =e, —€,,4 ; with 


uj 
a coefficient of +1 in the representation. A typical representation of aj is 


Ay = Ajp — Aye + Ay, —A,, +A,y 
= (€;-€n44)—(€e —Cnsk) + (Cp —€mis) 
—(e, —Cmis) +(e, —€ms;) 


= e; ems 7: 


A representation of the nonbasic vector a;; in terms of the basic vectors is illus- 


trated by the transportation matrix (tableau) in Figure 10.2. Note that the cell (i, 
J) together with the cells (i, 4), (4 4), (4, 5), (u, s), and (u, 7) form a cycle in the 
matrix. The cells (i, &), (4, k), (4 s), (u, s), and (u, /) form a chain in the matrix 
between cell (i, 4) and cell (u, 7). Other basic cells that do not appear in 
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Figure 10.2. Illustration of the representation of a,, in terms of the 


basic vectors. 


the representation of a, are not shown in Figure 10.2. Also, note that the signs 


of the coefficients alternate throughout the chain. 


Characterization of a Basis on a Transportation Tableau 


In Chapter 9, we saw that any basis B of A° corresponds to a rooted spanning 
tree and vice versa. In the sense of the transportation tableau, the interpretation 
of this fact is as follows. The basis is rooted, that is, it (necessarily) contains the 
root arc x,. The remaining (m + n — 1) basic variables correspond to some cells 


in the transportation tableau. The basis must be spanning, that is, there must be 
at least one basic cell in each row and in each column of the transportation 
tableau, or else, a row of B would have all zero elements (why?), which would 
disqualify it as a basis. Finally, the basis must be a tree, that is, the (m +n — 1) 
basic cells should not contain a cycle, or else we can write one column of B in 
terms of the others. 

Figure 10.3 shows a basis on the transportation tableau as well as its 
graph representation. The basic cells are identified with Bs on the transportation 
tableau and the corresponding arcs are depicted on the graph. Note that there is a 
unique chain connecting every pair of basic cells shown by the lines in Figure 
10.3, that is, there is a unique way of getting from any row or column of the 
tableau to another row or column while stepping along the basic cells. This 
corresponds to a unique chain comprised of basic arcs connecting any pair of 
nodes in the basis graph. For example, to get from origin | to destination 3, we 
step along the basic cells or arcs (1, 2), (2, 2), and (2, 3). This connectedness 
property is a consequence of the fact that the basis has (m + n — 1) basic arcs or 
cells and has no cycles. 

The basis is also lower triangular (after a suitable permutation of its rows 
and columns). Because the basis graph is a tree having at least two nodes, it has 
at least one end node that is not the root node, that is, not a node at which the 
root arc is incident. Since such an end node is one at which only one arc is 
incident, this corresponds to a row or column of the transportation tableau that 
has only one basic cell. In Figure 10.3 the origin nodes | and 3 and the 
destination node 1 are the ends of the basis tree. Let us now select the end 
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(S) 
© © 


@) ©) a 


Figure 10.3. Illustration of a basis on the transportation tableau and the 
transportation graph. 


O;, say, and permute the rows and columns of B so that its first row and column 


correspond to O; and x,>, respectively. Deleting O, and its incident arc in the 


graph, and hence, deleting row 1 along with the basic cell (1, 2) in the 
transportation tableau, the remaining graph or tableau represents another tree 
having one less node. Repeating this process, we obtain B in lower triangular 
form. The rows and columns identify the sequence in which the ends and their 
corresponding incident arcs have been considered. 


2 x92 Xo X93 %33 2a 


a 
0, 1 0 0 0 0 0 
Dy -1 -1 0 0 0 0 | 
B= D, 0 0 -1 0 0 0 | 
O) 0 1 I l 0 0 
O; 0 0 0 0 1 0 
D, 0 0 0 -1 =| l 


i 
Ln 2 ee —_ : 


Because of the triangularity of the basis, the systems of equations Bxz = b, wB 
= Cp, and By; = aj for computing the flows, the dual variable values, and the 


representation of a nonbasic variable column in terms of the basic variable 
columns, respectively, can be solved very simply and efficiently. The solution of 
these systems of equations on the transportation tableau, along with the accom- 
panying simplex operations are addressed next. 
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10.3 REPRESENTATION OF A NONBASIC VECTOR IN TERMS OF 
THE BASIC VECTORS 


We have determined that each nonbasic cell together with a subset of the basic 
cells forms a cycle and that the basic cells in this cycle provide the required 
representation for the nonbasic cell. Then we saw that the set of basic cells 
forms a spanning tree on the transportation matrix. We further know that there is 
a unique chain between every pair of cells in the tree; otherwise, cycles would 
exist. All of this suggests that to find the representation of a given nonbasic cell 
(i, 7) we can use the chain in the basic tree between some basic cell in row i and 
some basic cell in column /. This is essentially accurate except that not all basic 
cells in this chain (in the transportation matrix) are in the representation. 

To produce the proper representation for a given nonbasic cell (variable) 
we simply locate the unique cycle, in the basis graph, containing the arc associ- 
ated with the particular nonbasic cell. Then all of the basic cells of the 
transportation matrix associated with the arcs of the cycle in the graph are 
required for the representation of the nonbasic cell. The process of locating the 
representation directly on the transportation matrix is essentially the same, except 
that not all basic cells in the unique cycle are used. In this case, we use only 
those cells of the chain for which there is another cell of the chain in the same 
row and another cell of the chain in the same column, that is, the corners of the 
cycle. 

To illustrate, consider Figure 10.4. Suppose that we want to represent 
a4 in terms of the basic vectors. In Figure 10.5 the unique cycle of the graph is 


given by (1, 4), (3, 4), (3, 1), (1, 1). Deleting the nonbasic arc (1, 4), we are left 
with the unique basic chain (3, 4), (3, 1), (1, 1). As we already know, these are 
assigned alternating signs of +1 and —-1, giving the following representation: 


ay 4 = ay) — 83) +434, 
which we may verify by 


€) —€44.4 = (€] — C441) — (€3 —€ 441) + (€3 —€ 444). 


1 2 3 4 $ 


Figure 10.4. Illustration of finding the representation of a nonbasic cell. 


The Transportation and Assignment Problems §21 


Figure 10.5. The cycle in the graph associated with the basis of Figure 10.4. 


If we had sought the representation from the transportation tableau, we 
would first obtain the unique cycle (1, 4), (1, 2), J, 1), 2, 1, GB, 1), G, 3), GB, 
4). Other than the nonbasic cell (1, 4), the corners of the cycle are cells (1, 1), 
(3, 1), and (3,4). 

The representation is given below: 


As another example, we shall represent the vector a4, associated with the 


nonbasic cell (4,2) in terms of the basic variables. Tracing the cycle C in the 
transportation tableau, we get 


C= {, 2), (1, 2), 0, 1), 2, I, B,D, 3,3), (4, 3)}- 


The required basic cells are the corners (1, 2), (1, 1), (3, 1), (3, 3), and (4, 3). 
Hence, the representation is 


Ag? = Ajy2 — A)] + 43) — 833 + Ag3, 


which can easily be verified as correct. Pictorially, the representation is (ignor- 
ing other basic points) as follows. Although this appears to be two cycles, it is 
actually one cycle through the basic cells and cell (4,2), since the cell (3,2), 
where the lines cross, is not basic. 


ay, BD B tay 


+83 B. 


B —-a3, 
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The Role of the Artificial Variable in the Transportation Problem 


We note that the representation of a nonbasic vector involves only basic vectors 
associated with the unique chain through the basic cells. In particular, the 
artificial vector never becomes involved in any representation, and therefore, the 
artificial variable will always remain zero. This fact will allow us essentially to 
ignore the artificial variable in the application of the simplex method to 
transportation problems. 


10.4 THE SIMPLEX METHOD FOR TRANSPORTATION PROBLEMS 


The general steps in the application of the simplex method to a linear program 
are as follows: 

1. Find a starting basic feasible solution. 

2. Compute z;—c; for each nonbasic variable. Stop or select an enter- 


ing column. 
3. Determine an exiting column. 
4. Obtain the new basic feasible solution and repeat Step 2. 


We shall show how each of these steps can be carried out directly on the 
transportation tableau. 
Finding a Starting Basic Feasible Solution 


In Section 10.2 we produced a feasible solution for the transportation problem. 
However, the solution was not basic. While it would not be difficult to convert 
that solution into a basic feasible solution, we shall consider another procedure 
for obtaining a basic feasible solution. This method is called the northwest 
corner rule. During its process, as a variable x;; is assigned a value, we reduce 


the corresponding s; and d; by that value. Let the reduced values of s; and d; be 
denoted by s; and d /;» tespectively. In particular, to start with, $; = s; and d ja; 


Assuming that the total supply equals the total demand, beginning in cell 
(1, 1) we let 


x1 = minimum {Ss} di } 


and replace §, by $;—x,, and d, by d,—x,. Then, if s; > 0, we move to cell 
(1, 2), let 


x2 = minimum {§,,d>} 


and replace §, by $,—2x2 and d by dy —x,7. However, if $, = 0, then we 
move to cell (2, 1), let 


xX», = minimum {§,d,} 
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and replace S by $y — x, and dy by d, —X 1. Note that the case s, = d, produces 


degeneracy, because after computing x,,, the revised values s, and d, are both 
zeros. Hence, the basic variable x,, = 0. (Observe that in this case we could 
have alternatively moved to the cell (1, 2) and made the variable x,» basic at 


value zero.) In the general case, after assigning some variable x,» the value 

Xe = minimum {5;.dy}, 
we replace S$, by Sy —X,¢ and dy by dy —Xzy. If (k, = (m, n), we terminate. 
Otherwise, if 5, > 0, we move to the cell (k, £ + 1), which must exist (why?), 
and compute x; (¢,1) = minimum psa On the other hand, if s, = 0, we 
move to the cell (A + 1, 4), which must exist (why?), and compute x;,41,¢ = 
minimum {5;,,,d,}. Note that if §, =d, in the computation of x;,,, then in the 


latter case we will have the revised dy = 0 and we would obtain x;,,) as a 


degenerate basic variable. The process of assigning a variable the minimum of 
the remaining supply or demand, adjusting both, and moving to the right, or 
down, one cell at a time continues until all supplies and demands are allocated. 
Figure 10.6 illustrates how the process might work. 

The northwest corner rule produces exactly m + n — 1 basic, nonnegative 
x, variables. Each time an x is made basic and assigned some nonnegative 


value, either a supply or a demand constraint is satisfied. When m + n —- 1 
variables have been assigned values, then m + n — 1 of the constraints are 
satisfied. Noting that one of the constraints of the transportation problem is 
redundant, then all the constraints are met. 

The graphical structure of the basic cells is obviously connected and 
spanning. To demonstrate that the graph is a spanning tree and therefore a basis, 
it remains only to show that it contains no cycles. (Actually, this follows from 
the additional fact that the graph has m + n — 1 basic arcs.) Since, at each step, 
either the row or column index of the basic variable is increased by 1, it is not 


“TH 5} 
B B B $2 

TO « 

B 54 


Figure 10.6. Graphical illustration of how the northwest corner rule might 
allocate values to the variables. 
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Figure 10.7. Example of the northwest corner rule. 


possible to make a new variable in an earlier row or column basic—the only 
way to produce cycles. Therefore, the northwest corner method produces a basic 
feasible solution. An alternative procedure, called Vogel’s approximation method, 
is described in Exercise 10.25. 

To illustrate, consider the transportation tableau of Figure 10.7 where the 


supplies and demands are indicated. We first let x,, = minimum {51,d)} = 15, 
decrease 5, and d, by x1, = 15, and move to cell (1, 2), since 5; > 0. Next, let 
X}2 = minimum {51d} = minimum {15, 20} = 15, decrease §, and d>, and 
move to cell (2, 2), since 5; = 0. This process is continued until all supplies and 


demands are satisfied. Notice that we do have the required number of basic 
variables, namely 7 = m + n— 1. The blank cells are nonbasic and the associated 
variables have zero values. 


Computing z; — cj for Each Nonbasic Cell 


Given a basic feasible solution, our next task is to determine whether that 
solution is optimal or to select an entering variable. Now, 


Zi — Cy = CRY y — Cy- 
We have shown how to obtain the components of Yy> that is, the coefficients of 
the representation of ayy in terms of the basic vectors. Since Yi consists of 1, —1, 
and 0 elements, then c BYij is calculated by simply adding and subtracting the 
costs of some basic variables. To illustrate, consider Figure 10.2. The z, — Cy 
value for the nonbasic variable x, is given by 

Zi — Cy = (Cj — Cus + Ces — ek + Ck) — Cy: 


Using the data of the example in Section 10.1 and the basis indicated 
below, we get the following result: 
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221 ~ C2] =4-54+3-2=0 


297 —C97 = 7-54+3-4=1 
and x9 is a candidate to enter the basis. 
The optimality criterion for the transportation problem is given by 2, —cj 
< 0 for each nonbasic variable x,;. A given cell (k, ¢) is a candidate to enter the 
basis if zz» —cyp > 0. . 


The foregoing procedure for calculating z;; —c, utilizes the form 


= =] = 
Zi —Cyj =CpB ay — Cp = BY — Cy: 


The vector y;; is determined by constructing the unique cycle through cell (i, /) 


i 
and some of the basic cells, as discussed earlier. Hence, this method is some- 


times called the cycle method. Note, however, that calculating zy — Cy can be 


alternatively performed as follows, which is the preferable method. 


a -l 
23; 7 Cy =CpB ay —¢ 


ij = way —C 


y ij 
Let w,, i,.... m, be denoted by u; and Wins js J = 1,..., 2, by Vj. Then the dual 
vector w is given by 

W = (Uy 5005 Uyg eV o-009 Vy) 


Since a; has a | in the ‘th and a —1 in the (m + /)th position, then wa,; = u;—v 


] i 
Hence, 2 —¢y =u; —Vv; —¢j. This method of calculating z; —cj is thus called 


the dual variable method or the (u;,v 'j-method. 
Because the dual vector is given by w =c 2B, then w is the solution to 
the system 
wB=cp, 
where 
B= (8 pgs. Ast min) 


CB = (Cyg ost sq) 
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ne : 
and Apgo-sAsy are mtn 1 basic columns, Crass 


cost coefficients, e,,,, is the artificial column, and c, is its cost coefficient. 


Cy are their corresponding 


Because the value of the artificial variable will never vary from zero, as 
previously shown, the value of c, does not matter. For convenience, we shall 


select c, = 0. Because B is triangular, we have an easy system to solve. The 
system 


(Uy 5 -+2sUygs Vp se Vy MA pg >-8s1>€mtn) = (Cpg yevea 5p 50) 


is equivalent to (since aj; = €; —€n, ;): 


P q Pq 
Us — Vy = Cot 
Vy, = 90. 


The foregoing system has m + n variables and m + n equations. Utilizing the 
concept of triangularity, we back—substitute the value v, = 0 into each equation 


where v, appears and solve for a uw—variable. Using this newly found u- 


variable, we back—substitute to find some v—variable value(s), and so on. 
As an illustration, consider the example problem of Section 10.1 with the 
basis as indicated. The last dual variable v3 receives the value zero from the 


artificial column. 


Using the artificial column > 3 =0 
Using the basic cell (2,3): u.—-v3=3 =u =3 
Using the basic cell (1,3): u-v3=5 2m =5 
Using the basic cell (1,2): uj-v2 =7 vy =-2 
Using the basic cell(1, 1): u-y,=4 >ye=l. 


Only the basic cells are used to solve for the dual variables. Given the w—vector, 


we may compute z,—c, for each nonbasic variable in order to determine an 


entering column. In particular: 


Z21 ~€2) =UQ-V,— C2] =3-1-2=0 
292 —C€2 = uy — V2 — C72 =34+2-4=1 
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and, again, we see that x) is a candidate to enter the basis. 
Determination of the Exiting Column 


Once a column (cell), say, (A, £), has been selected to enter the basis, it is an 
easy matter to determine the exiting column. Recall that the coefficients in the 
basic representation for that column are the negatives of the rates of change of 
the corresponding basic variables with a unit increase in the nonbasic (entering) 
variable. Thus, if the entry in column y;z, corresponding to a basic variable is 
~], then the basic variable will increase at the same rate as the nonbasic variable 
Xz¢ increases. If this entry is +1, then the basic variable will decrease at the 
same rate as the nonbasic variable x, increases. 


Let Le be the value of Xj in the current solution and let A be the amount 


by which the nonbasic variable x, increases. Because each component of yz, is 
either 1,—1, or 0, then the usual minimum ratio test gives 
A= minimum {iy: basic cell (i, 7) has a +] in the representation 
of the nonbasic cell (4, 2)}. 


Given A, we proceed to adjust the values of the variables around the cycle by 
this amount, according to the sign of the coefficient in the representation. For 
the example of Section 10.1 with the basis indicated below, as x2 enters the 


basis, we get the following result: 


1 


A= minimum {x,7,x23} = minimum {10,20} =10. 
The new solution is given by 


X42 =X2-A=10-10=0 (leaves the basis) 
X13 = %3+A=54+10=15 

X93 X93 -A = 20-10=10 

x22 A =10 

X; =15 (unchanged). 


The new basis is identified in the following tableau: 


528 Chapter 10 


10.5 ILLUSTRATIVE EXAMPLES AND A NOTE ON DEGENERACY 


Example 10.1 


Consider the transportation problem indicated by the data of Figure 10.8. The 
number in the upper left-hand corner of each cell is the cost associated with the 
particular variable. The starting basic feasible solution produced by the 
northwest corner method is illustrated in Figure 10.9. 

Beginning with (1, 3), we price out each of the nonbasic cells by the 
cycle method: 


Zi t9 = 02-18412)-13 =+7 
214 -Ci4 = (12-18+16)-8=2 
215 —C15 = (12-18+16-14+10)-14=-8 


245 —C4g = (10-18 413)-12 = -7. 


1 2 3 4 5 6 Ry 


, (oy 27 318 T L14] [197 | 1 

5 (15] [18] [12] [16] | 19] [20] | 29 

3 {177 [16] 1137 [14] [10] [18] | 39 
T T T T 

4 [19; [18] 207 [217 [12] [137 | 44 

d, 10 il 13 20 24 15 


Figure 10.9. The northwest corner basic solution. 
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The current values of the basic variables and the tree are shown in Figure 


10.10; the z;;—cj values for the nonbasic variables are the circled values. 


Because 2; mee 0 for the basic variables, these are not indicated. Note that 


the Zi — Cy 


follows. First solve for the dual variables (their values are summarized in Figure 
10.10). 


values could have been alternatively calculated more efficiently as 


artificial column > vy, =0 

basic cell (4,6): ug—v6 =13) Sug =13 

basic cell (3,6): 3 -—Ve =18 =u, =18 

basic cell (3,5): u3-v5 =10 v5 =8 

basic cell (3,4):  u3-vg=14 >vwy=4 

basic cell (2,4): uy. —vg =16 => uy = 20 

basic cell (2,3): uw. -v3 =12 > v3 =8 

basic cell (2,2): uy —vy =18 =v, =2 

basic cell (1,2): up—-vy =12 2m =14 

basic cell(1, 1): u-y=10 =>y=4. 

Then, 2; —¢ =u; -v; —¢j. For example, 214 -—cj4 =u —V4 —C14 = 14-4-8 
= 2. Figure 10.10 indicates that the maximal z,;—c, value is zj4—-c4 = 2. 


Therefore, x,4 enters the basis. Referring to Figure 10.10, we see that 


Aq = Ay2 — 822 + Arq 
and the corresponding cycle is as follows: 


y=4 w=2 


Figure 10.10. The (24 — ¢4)—values for the nonbasic cells. 
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x4 = minimum {%,7,%24} = minimum {8,6} =6 
2 = 8-6= 2 

x42 = 3+6=9 

X94 = 6-6=0 (leaves the basis). 


The new basis and the values of the basic variables are summarized in 


Figure 10.11. Using the dual variable method, z,—c, is calculated for each 


nonbasic variable. 
As indicated by the circled entries in Figure 10.11, x32 can be selected as 


the entering variable. The cycle associated with cell (3, 2) is as follows: 


0,2) 4) 


From this we obtain 


x39 = minimum {%,>,%34} = minimum {2,14} = 2 
X72 =2-2=0 (leaves the basis) 

14 = 64+2=8 

%34 = 14-2=12. 


The new basic feasible solution and the new z,;—cj values are given in 


Figure 10.12. Examining the z,;—cj entries, we find that x2, is enterable. The 


associated cycle is as follows: 


(1, 1) (1, 4) 
Q, 1)-~ (2,2) 


(3, 2) 


(3,4) 


From this we obtain 


uy, =12 


uy =18 


uz =18 


ug =13 


y=2 wW=0 Ww=6 w=4 v5=8 ve =0 


Figure 10.11. Second basic feasible solution. 
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y=2 wW=2 wy=8 w=4 


u, =12 


uy =17 


43 =18 


ug =13 


Figure 10.13. Fourth basic feasible solution. 


X71 = minimum {X, 1,%34,X22} = minimum {10,12,9} =9 


X11 =10-9=1 
X14 = 8+9=17 
X34 =12-9=3 
3 = 24+9=11 


X97 =9-9=0 (leaves the basis). 


Figure 10.13 presents the new basic feasible solution and the new 


— Cj. Because z;, —c;, < 0 for each nonbasic variable, the indicated solution is 


Z 7s 


ij ij 


optimal. 
Example 10.2 
(Degeneracy) 


Consider the example of Figure 10.14. Applying the northwest corner rule to 
this example we obtain the following sequence of calculations: 


x, = minimum {5,,d,} = minimum {20,10} = 10 


§ =20-10=10, d, =10-10=0 


§32 Chapter 10 


x) = minimum {$,,d)} = minimum {10,10} = 10 


§,=10-10=0, d,=10-10=0. 


At this point we may move to either (2, 2), or to (1, 3). Suppose that we move to 
(2, 2). 


x5) = minimum {8,4} = minimum {30,0} = 0 
$ =30-0=30, d,=0-0=0 
X93 = minimum {8,43} = minimum {30,20} = 20 
$) =30-20=10, d; =20-20=0 
Xo4 = minimum {8 ,d4} = minimum {10,50} = 10 
§ =10-10=0, dy =50-10=40 
x34 = minimum {83,44} = minimum {40,40} = 40 


$3 =40-40=0, dy =40-40=0. 


y=-10 w.=-1ll yw=4 wW=0 


Figure 10.15. Initial (degenerate) basic feasible solution. 
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All other xj;—variables are nonbasic and are assigned zero values. The initial 


basic feasible solution is given in Figure 10.15. As required, there are m+n— 1 
=3+4- 1 =6 basic variables forming a spanning tree. Note, however, that the 
basic feasible solution is degenerate since the basic variable x2. = 0. For each 


nonbasic variable we calculate z;; —cj by the dual variable method. The values 


are displayed in circles in Figure 10.15 for the nonbasic variables. Because 
23) —C3, = 1, then x3, enters the basis. The corresponding cycle is as follows: 


(1,.1)}— (1,2) 


(2, 2) 


(2, 4) 


x3) = minimum {x} ,%2,%34} = minimum {10,0,40} = 0 
x}, =10-0=10 
X21 =10+0=10 
X27 =0-0=0 (leaves the basis) 
X4 =10+0=10 


X34 = 40-0= 40. 


Note that x». leaves the basis and x3, enters the basis at a zero level. This 
results in the same extreme point solution, but a different basis. The new basis 
and the new (2; —¢,)-values for the nonbasic variables are shown in Figure 


10.16. Since zy oy 0 for each nonbasic variable, the current solution is 


optimal. 
Notice that in this example we have 


20 = sy =d,+d, =10+10 


1 2 3 4 


y=-9 vw.=-10 w=4 wW=0 
Figure 10.16. Optimal basic feasible solution. 
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or, in other words, the sum of a subset of the supplies equals that for a subset of 
the demands. We shall show that this is always true when degeneracy occurs in 
the transportation problem. 


A Necessary Condition for Degeneracy in the Transportation Problem 


Suppose that at some iteration of the transportation algorithm we obtain a 
degenerate basic feasible solution, such as that shown in Figure 10.17. Deleting 
one of the degenerate cells disconnects the tree into several components as 
shown in Figure 10.18. Summing the supply constraints over the variables in 
one of the components, say, C; , we obtain 


Summing the demand constraints over the variables in the same component, we 
get 


Lx =Ld;. 
C ” C 


Together, these equations imply that 


Thus, a necessary condition for the presence of degeneracy is that a proper 
subset of the rows and columns have their total supply equal to their total 
demand. Therefore, there is no degeneracy (and cycling) if no such proper 
subset exists (see Exercise 10.21). 


Figure 10.18. Components created by deleting a zero basic cell. 
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10.6 THE SIMPLEX TABLEAU ASSOCIATED WITH A 
TRANSPORTATION TABLEAU 


We have all of the information available to construct the simplex tableau 
associated with a transportation tableau, if we so desire. In Section 10.3, a 
method was described for calculating the updated column vector y,- In Section 


10.4, we saw how to calculate Zy — Cy for a nonbasic variable Xz. This informa- 


tion together with the basic solution provide all the necessary entries in the 
simplex tableau. 

As an example, consider the transportation problem given by the 
following data: 


CF 1 2 3 5; 
“0: rt 20] 55 
20 

15 15 10 


j 
Figure 10.19 presents the initial transportation tableau and the associated 
simplex tableau (including the artificial variable). Examining either tableau, we 
see that x2, enters the basis and x2 leaves. We ask the reader to verify the 


entries in the simplex tableau by generating the (2;; — cj )— and the y,—values 
from the transportation tableau. 

10.7 THE ASSIGNMENT PROBLEM: (KUHN'S) HUNGARIAN 
ALGORITHM 


An important special case of the transportation problem is the case where m = n, 
each s; = 1, and each d,; = 1. This special case is called the assignment 


problem. As an example, suppose that we have m individuals and m jobs. If 


Figure 10.19. An initial transportation tableau and the associated 
simplex tableau. 
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individual i is assigned to job /, the cost incurred will be cy. We wish to find 


the minimal cost assignment or a one-to-one matching of individuals to jobs. In 


each basic feasible solution x, = 1 means that individual i is assigned to job j, 


and x, = 0 indicates that individual i is not assigned to job j. 


Because the assignment problem is a special case of the transportation 
problem, we could apply the transportation algorithm developed in this chapter. 
Note, however, as will be discussed in more detail, that the constraints of the 
assignment problem admit exactly m positive variables at each basic feasible 
solution. The number of basic variables is 2m — 1. Thus, if the transportation 
algorithm is used, we would have m — 1 basic variables at zero level, leading to 
a highly degenerate problem. A partial resolution of this difficulty is addressed 
in Section 10.8. In this section, we shall exploit the special structure of the 
assignment problem to derive a competitive primal—dual type of algorithm. 

A mathematical model for the assignment problem is given as follows: 


mm 
Minimize yy CyXij 
i=l j=l 
. m 
subject to > xy = if i=1,...,m 
j=l 


—L xy =-1, j=l,....m 
x, =Oorl, i,j=l,....m. 
In matrix form, the assignment problem can be stated as follows: 


Minimize cx 
subjectto Ax=b 
xy =O0orl, i,j=l,...m, 


where KS (4 aah tan) : A is a 2mxm matrix whose (i, /)th 


column is ay =C; —Cmsj for i=1,..., m andj = 1,...,m, and b= (1,-1)', where 1 


iS a row—vector of m ones. Thus, we see that A is the same constraint matrix as 
that for the transportation problem. The underlying graph is bipartite, and so the 
assignment problem is sometimes referred to as a “minimum weighted matching 
problem on a bipartite graph,’ with weights cj. Applying the total 
unimodularity property of A, we know that an optimal basic feasible solution to 
the assignment problem with the constraint x, = 0 or 1 replaced by x 2 0 will 
be all integer. Furthermore, as a result of the constraints, no xij value can exceed 
1. Hence, all x, values will be either 0 or | in an optimal solution to the linear 
program. This permits us to replace the constraint x; = 0 or | by the constraint 


x, 20. Thus, we obtain the following: 
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Minimize ex 
subjectto Ax = b 
x 2 0. 
It is interesting to note that any transportation problem can be 
equivalently transformed into (its special case) an assignment problem. This is 
readily, although inefficiently, accomplished by making s; copies of origin O,, 


each with a unit supply for i = |,...,m, and similarly, by making d; copies of 
destination D is each with a unit demand for j = 1,...,.7. The problem hence becomes 
one of matching the d=)ijs; = 2 jd; units of supply and demand, where the 


cost of matching a unit of supply from a copy of O; to a unit of demand at a 
copy of D; is Cy. 


Observe that the assignment polytope has m! extreme points. Viewing the 
assignment constraints as matching m “origins” O;, i= 1,...,m, to m destinations 


De J= 1,...,m, there are evidently m! possible matchings, each corresponding to 


some permutation of the destinations being matched with the origins Q),...,0,,. 
For each matching, choosing the corresponding x, variables that equal | as 


basic variables, we obtain m basic variables that are unity. Figure 10.20a depicts 
the situation for some matching with m = 4. To complete the basis, we need to 
select (m — 1) (degenerate) variables or arcs so that the resulting graph is 
connected and hence, a tree. (Note that a connected assignment subgraph with 
(2m — 1) arcs cannot contain a cycle.) Figure 10.20b shows one choice of 
degenerate basic arcs. Because every feasible integer solution is a matching, and 
every matching is an extreme point of the assignment polytope, the assignment 
polytope has m! vertices. Moreover, every basic feasible solution has (m — 1) 
degenerate arcs, as noted earlier. In fact, it can be shown that every vertex has 


(2”1)\(m™~) bases representing it. Furthermore, given any pair of extreme 
points x #y, either x and y are adjacent vertices or there exists a third extreme 
point z such that x and z are adjacent and so are y and z (see Exercises 10.41 
and 10.42). 

The Dual Problem 


The dual of the assignment problem, with the nonnegativity restrictions replac- 
ing the 0 — | constraints, can be written as follows: 


1 1 
@) Z Or? ® 
@—-&) Orme, 
(0;_1_>0,) 0, —1_=D, 


(a) (b) 
Figure 10.20. Matchings and basic feasible solutions. 
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i=l j=l 
subject to uj —V; Scy, i,j=l,....m 


Uj,V; unrestricted, i,j =1,....m. 


ae m m 
Maximize Lu->d vj 


The complementary slackness conditions are given by 
(cj —uj;+v,)xjy =0, RL J=1,..m. 


Thus, if we can find a set of feasible values for the x, u, v, variables that satisfy 
complementary slackness, the resulting primal—dual solution will be optimal. 
A feasible dual solution is given by 


uj; = minimum {cj}, i=l,....m 
ls j<m 
y; =—minimum {cj —4;}, JHl.asm 
lsi<m 


From this we see that #; is the minimum c;; in row i and ¥, is the negative of the 


minimum c,, —u,; in column. 


i 
The Reduced Matrix 


Consider a reduced cost coefficient matrix where cj, is replaced by ¢j = cj — 


u; +v,;. In other words, the reduced cost matrix is obtained by first subtracting 


from each row the minimum in that row, and then on the resulting matrix 
subtracting from each column the minimum in that column. The reduced matrix 
will have a zero in every row and column, and all of its entries will be 
nonnegative. The reduced matrix is actually the matrix of dual slack variables 
(why?). 

Suppose that we can find a feasible set of x;;—variables such that each 


xy with value | is associated with a zero cell of the reduced matrix. Then, by 


complementary slackness, we can conclude that we have an optimal solution. 


What, then, constitutes a set of feasible xy ~—values? Reviewing the constraints 


of the assignment problem, it is clear that we must have exactly one xj; in each 


row equal to | and exactly one x, in each column equal to 1. Thus, in a feasible 


solution, there will be exactly m of the x,—variables equal to 1, the rest being 
zero. 


Let us illustrate the foregoing ideas with an example. Consider the 
following cost coefficient matrix for an assignment problem: 
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I 2 3 4 Row minimum 
By, 23 2 5 4 2 
2| 0 | I 2 3 0 
3; 4 I -I 3 -l 
4; 2 | 5 3 4 2 


Subtracting the row minimum from each element in the row, we get the follow- 
ing tableau: 


1 2 3 4 
lio 0 3 aod 
2 0 1 2 3 
B i 
3,5 2 0 4 | 
4) 0 3 1 4 
0 0 0 2 Column minimum 


Subtracting the column minimum in the new matrix from each element in the 
column, we get the reduced matrix as follows: 


Now, if we let x} =x}; =x33 = 44 =1 and if we let all other xj-values be 
zero, then we have a feasible solution with positive x,—values associated with 


the indicated zero cells of the reduced matrix, thus producing an optimal 
solution. 

It is not always so easy to find an optimal solution. Take, for example, 
the following cost matrix: 


ne ae: 
i ee ee 
2] 4 | 2 | 1 {@e%) 
3} 2 | 6 | 5 | 


— 


wlolnN;]; ts 
wloln!] w 
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Here, it is not possible to set three of the x,;—variables equal to 1 such that all 


xij 


three positive x,;—values occur in zero cells and no two positive x, —values 


occur in the same row or column. 
A Partial Solution 


Notice that for the foregoing reduced matrix, the maximum number of xj 


variables associated with the zero cells, which can be set equal to 1 without any 


two positive xj—values occurring in the same row or column, is two. For 


example, we might let x,; = x22 =1, or x1) = 23 =1, or x3) =X 2 =1, or %3) = 


X23 = 1. In this case, the maximum number of cells having zero ¢;;—values such 


that no two cells occupy the same row or column is two. The corresponding 
cells are called independent. Notice also that if we were to draw a set of lines 
through the rows and columns to cover the zeros so that there is at least one line 
through each zero, the minimum number of such lines for this matrix is two: A 
line through column 1 and a line through row 2. 


We see in this example that the maximum number of independent zero cells and 
the minimum number of lines required to cover the zeros are equal. This result, 
which is true in general, is given by the following theorem. We shall not prove 
this theorem here. (In Chapter 12, Exercise 12.17 asks the reader to show that 
this theorem is a special case of the maximal flow—minimal cut theorem for 
networks. At that time we also suggest a method for systematically finding the 
required number of lines. Also, see Exercise 10.38 and Exercise 10.39.) 


Theorem 10.1 


The maximum number of independent zero cells in a reduced assignment matrix 
is equal to the minimum number of lines to cover all the zeros in the matrix. 


Modifying the Reduced Matrix 
Suppose that we have not yet obtained an optimal solution, that is, we cannot 
find a feasible set of positive x,—values associated with the zero cells of the 


reduced matrix. Consider the covered matrix obtained by covering the zeros in 
the reduced matrix by using the fewest number of lines. Let & be the number of 
lines required. Also, let S,. = {i,i,...} be the set of uncovered rows and S, = 


{jy jz.-} be the set of uncovered columns. Define S,=M-S, and S. = 
M-S,., where M= {1, 2,..., m}. Finally, let p be the number of rows in S, and 
q be the number of columns in S,. Then, k= (m—p) + (m— 4). 
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Let cg be the minimum uncovered element, that is, 


cg = minimum {ey} >0. 
icS, 


jeS, 


It can be easily demonstrated that a new dual feasible solution is given by 


; =u; +c, ieS, 
u; =0;, ieS, 
Vv, =;, Jes, 
vip =v; +Cq, JeES,. 


In the reduced matrix having elements c;;—u;+v,;, this is equivalent to sub- 


tracting cy from each uncovered row and adding cg to each covered column. 
Another way to view this is that cg is subtracted from each uncovered element 


and added to each twice-covered element. The new reduced cost coefficient 
matrix has nonnegative elements and a zero in every row and column (why?). 

For the previous 3 x 3 matrix, we have cg = minimum {2, 5, 3, 3} = 2 
and the new reduced cost matrix is given by: 


Notice that now a feasible set of Xj —values exists having positive xj —values 


associated with the zero cells (zero dual slack variables). 

Note that primal feasibility is attained, dual feasibility is maintained 
(since the entries in the reduced cost matrix are nonnegative), and finally, 
complementary slackness holds true (since xj = 1 only if the corresponding 


dual slack is zero). Thus, the Karush-Kuhn—Tucker conditions are satisfied and an 
optimal solution given by x17 = x73 = x3, =1 (with all other Xj —values equal to 
0) is at hand. 

Summary of the Hungarian Algorithm 

The algorithm developed in this section may be summarized as follows. 


INITIALIZATION STEP 


For each row of the cost matrix, subtract the minimum element in the row from 
each element in the row. For each column of the resulting matrix, subtract the 
minimum element in the column from each element in the column. The result is 
a reduced matrix. 


542 Chapter 10 


MAIN STEP 


1. Draw the minimum number of lines through the rows and columns to 
cover all zeros in the reduced matrix. If the minimum number of 
lines is m, then an optimal solution is available. Otherwise, go to 
Step 2. 

2. Select the minimum uncovered element. Subtract this element from 
each uncovered element and add it to each twice—covered element. 
Return to Step 1. 


An Example 
Consider the following cost matrix: 
1 2 3 4 > 
1 2 3 5 1 4 
2) -i 1 3 6 2 
Bi) earl ae | ae [ieee silt Sq 
4 1 3 4 1 4 
5 7 1 2 1 2 


The reduced matrix is as follows: 


Here, the minimum number of lines to cover all zeros is three. The minimum 
uncovered element is 1. Subtracting this from each uncovered element, and 
adding it to each twice—covered element, yields the following reduced matrix: 


eo Ve wa 7s 
1 | 1 D ; 
2 | | 1 2 
Si ieee 3 
4 | ia 1 
54 a: 


Again, we do not have an optimal solution at hand. The minimum uncovered 
element is 1. Subtracting 1 from each uncovered element, and adding it to each 
twice—covered element, leads to the following reduced matrix: 
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In this matrix, an optimal solution is given by 24> = x31 = X35 = X44 = 53 =1 
and all other Xj —values equal to zero. 

In Exercise 10.40 we ask the reader to show that the Hungarian method 
for assignment problems is precisely the primal—dual algorithm applied to the 
assignment problem. 


Finite Convergence of the Hungarian Algorithm 


Whenever we cannot find a feasible set of x,,—values having ones associated 


with the zero cells of the reduced matrix, we repeat the process of drawing lines 
and adjusting the reduced matrix. We show below that we can only do this a 
finite number of times before an optimal solution is found. Clearly, an optimal 
solution can be found if all reduced costs become zero. To show finiteness we 
note that the reduced costs are always nonnegative and that 


LLGy-UUGy = 1s Gj - oH) oe (Cj — i) 
ij ij oS 


+ > & -Gj)+ Xd (;-G) 
Gay! Uwe 


yk 04 rt Oe oo) 
GS) Gs) CS) Gs) 


pqcg —(m— p\m—|q)cg 
m(p+q-—m)cy. 


But p + q is the number of uncovered rows and columns, so that 


p+q-m=(2m—-k)-m=m-k, 


where & is the total number of covered rows and columns. By Theorem 10.1, & is 
also the maximum number of independent zero cells. In particular, k < m because 
otherwise, we would have had an optimal solution at the last iteration. 
Therefore, 


Lue; — Cj) =m(m —k)co 
PJ 


is a positive integer provided that the original cost matrix consists of integers. 
Because the entries in the reduced cost matrix are always nonnegative by 
construction, and because the sum of the entries is reduced by a positive integer 
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at each iteration, the algorithm stops in a finite number of steps. At termination 
we have an optimal solution since the Karush-Kuhn—Tucker conditions hold 
true. 

As a final insight with respect to the Hungarian method, consider the 
following variation in the method. Suppose that at each step of covering the 
zeros of the reduced matrix, in lieu of using the fewest number of lines k, we 


cover the zeros using some k < m lines, if possible (else, an optimal solution is 
at hand). As before, then, let cy be the minimum resultant uncovered element 


and suppose that we perform the same type of reduction operation. From the 
foregoing discussion or the corresponding dual solution adjustment, we have 
that the difference between the new and the previous dual objective values is 
given by 


[D¥ -LV,]-[Ea,-E5,] 
i J i J 


= 01S, |~co| 8. |=colm-| 5, |-] 5. p =co(m—K) 21. 


Hence, we obtain a dual ascent by at least a unit, and since the dual objective 
value is bounded above, we will converge in a finite number of iterations. 
Observe, however, that in order to maximize the dual ascent at each iteration, it 


is not simply sufficient to select the minimum number of lines k = k to cover 
the zeros of the reduced matrix because the value cg is also governed by this 
covering process. That is, we can try and devise a mechanism to cover the zeros 
using k <m lines, whenever possible, such that the resultant product co(m—k) 


is maximized in order to accelerate the convergence process. The Notes and 
References section cites some improved modifications of the Hungarian method 
conducted along these lines. 


10.8 ALTERNATING PATH BASIS ALGORITHM FOR 
ASSIGNMENT PROBLEMS 


We have seen that every basic feasible solution to the assignment problem has 
degeneracy of order (m — 1). In fact, it has been empirically observed that 
typically 90 percent of the pivots used to solve the assignment problem by the 
usual primal simplex algorithm are degenerate when m > 500. This should not 
be surprising since we know that every non—optimal vertex is either adjacent to 
an optimal vertex or is adjacent to a neighbor of this optimal vertex. Hence, it 
stands to reason that if we devise a primal simplex algorithm that circumvents a 
sizeable portion of the degenerate pivots, then a significant savings in 
computational effort will result. The alternating path basis algorithm is a step in 
this direction. This algorithm is a special case of the strongly feasible network 
simplex algorithm discussed in Chapter 9, in which we additionally designate an 
origin node as the root node. For the assignment problem, a strongly feasible 
basis that has an origin node as a root node is called an alternating path basis. 
Because the algorithm encounters only strongly feasible bases, that is, bases in 
which all degenerate arcs point toward the root, it automatically circumvents 
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several bases representing the extreme points of the assignment polytope. As a 
result, this procedure yields about a 25% savings in pivots over the usual 
network simplex algorithm. 

Figure 10.20b shows an alternating path basis that is a chain graph. Such 
a basis can always be used to initialize the algorithm, given any choice of a 
starting one-to-one matching. (Exercise 10.47 indicates why this may be an 
advantageous starting point in the absence of any other information.) Figure 
10.21a displays a typical alternating path basis. Figure 10.21b exhibits a basis 
representing this same extreme point that is nof an alternating path basis (why?). 
Note that the name “alternating path” arises from the fact that the arcs having 
flows of one and zero alternate on the chain from any node to the root node. 
These arcs are respectively called one-arcs and zero—arcs. Observe, however, 
that not any feasible basis having an origin node as a root node and having one— 
arcs and zero—arcs alternating is an alternating path basis. For example, consider 
the basis obtained (with m = 5) in Figure 10.21b by removing O;, D3, O7, and 


D, along with their incident arcs. In fact, it can be easily shown (see Exercise 


10.45) that a feasible basis is an alternating path basis if and only if all one—arcs 
are pointing away from the root. This holds if and only if an origin node is a root 
node having one incident basic arc (other than the root arc), and all other origin 
nodes have exactly two incident arcs. This gives a special structure to an 
alternating path basis that can be algorithmically exploited. 

The algorithm using alternating path bases follows exactly the strongly 
feasible tree network simplex procedure of Chapter 9. However, some speciali- 
zations are possible. For example, observe that a pivot with entering arc 
(O;,D;) is nondegenerate if and only if D; is a successor of O,, that is, O; lies 


on the chain from D; to the root node. Whether the pivot is degenerate or not, 


the leaving arc, by the strongly feasible tree rule, is the one incident at O; in the 
basis equivalent chain or in the cycle that is formed by adding (O;,D;) to the 


basis graph. By noting that each origin node has a unique immediate successor, 
which is the destination node assigned to it, the tree may be stored by 
considering each such origin—destination pair as a single “married” unit, thereby 
reducing the storage and updating requirements by roughly one—half. We ask the 
reader to explore these specializations in Exercise 10.50. 

Computationally, the primal—dual algorithm of the previous section 
dominates the alternating path basis algorithm for totally dense problems when 
the cost parameters are in the range 0-100. However, unlike the primal-— 
simplex—based approaches, the primal—dual algorithm is quite sensitive to 
ranges in cost coefficients, as evident from its computational complexity. (It is 
also instructive to examine its finite convergence arguments.) Hence, the 
primal-simplex—based approaches dominate this algorithm for cost ranges 
exceeding 0-1000. Nonetheless, as the papers cited in the Notes and References 
section indicate, primal—dual types of algorithms do provide competitive 
alternatives to primal—simplex—based approaches for the assignment problem. 
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10.9 A POLYNOMIAL-TIME SUCCESSIVE SHORTEST PATH 
APPROACH FOR ASSIGNMENT PROBLEMS 


In this section, we will briefly present a polynomial-time algorithm for assign- 
ment problems based on solving a succession of shortest path problems. 
Although specialized polynomial-time techniques for solving shortest path 
problems on digraphs that do not contain any negative cost circuits will be 
discussed later in Chapter 12, we shall be content here to assume that a 
procedure is available that polynomially produces a basic optimal solution to a 
shortest path network flow problem as defined later. Computationally, when the 
procedure described in this section is used in conjunction with an efficient 
shortest path algorithm, it has been found to run up to seven times faster than the 
alternating path basis algorithm of the previous section. 

Let us present the fundamental algorithmic strategy of a successive 
shortest path approach using an example with m = 3 and with cost coefficients 
as given below: 


Suppose that at any intermediate step of the procedure, we have at hand a partial 
assignment solution in which some origin-destination pairs are tentatively 
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Figure 10.21. Alternating path basis for assignment problems. 
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matched. Let the sets UO and UD, respectively, contain the unassigned or active 
origin and destination nodes, and let UO and UD be their respective 


complements. To begin with, UO and UD are both empty. Let x denote the 
present partial assignment solution. Hence, x = 0 at the initialization step, and 
whenever an origin—destination assignment is made, the corresponding 
component of x will be set equal to one. Also, assume that currently 


| UO | = | UD | 21 so that x is infeasible to the assignment problem. 


Given a partial solution x, we first execute the shortest path step. For 
this purpose, let us construct a special network from the assignment graph as 
follows. For convenience, let the origin node O; be indexed as i for i = 1,...,m, 


and let the destination node D j be indexed as m + / for j = 1,..., m. Create an 


additional dummy root node 0 and connect this to each origin node in UO using 
a zero cost arc. (Let the flow on this arc be represented by the nonnegative 
variable z;, for i ¢ UO.) For each (O;,D;) pair for which x; is currently unity, 


create the reverse arc (m + /, i) and let the cost on this arc be —Cy. (Let the flow 
on this arc be represented by the nonnegative variable y,; =1—xj; note that 
currently, we therefore have y,; = 0.) All the other (O0;,D ;) ares in the 
assignment graph for which Xj = 0 are present as the usual arcs (i, m + /) in this 
new graph with the same variable x,; and cost c;;. Next, place a supply of | UD | 


=|UO| at node 0, and place a demand of one unit at each of the destination 
nodes m + j for D; UD. Let all the other nodes be transshipment nodes. The 


resulting network flow problem can be solved as the shortest path problem 
SP(x), which seeks a shortest path from node 0 to all the origin and destination 


nodes in this network (not just to D; « UD). Exercises 10.53 asks the reader to 


show that SP(x) actually results from relaxing the assignment problem by 
aggregating the unassigned origin node constraints; using the transformation Vij 
= 1 — x; for each (O;,D;) pair for which x, = 1; and tentatively relaxing the 
nonnegativity restriction on the latter x,—variables (i.e., not imposing yj < 1 in 
SP(X)). 

Note that SP(x) is feasible, (i.e., a path exists from node 0 to each of the 


origin and destination nodes in the underlying network for SP(x)), since we are 


considering a complete bipartite assignment network problem. (For a problem 
on a non—complete bipartite graph, if SP(x) is infeasible, then so is the original 


assignment problem—see Exercise 10.46.) Hence, if SP(x) is not unbounded, 


that is, it does not contain any circuit having a total negative cost, then it has an 
optimal solution. This is clearly so for the starting problem SP(X) with x = 0, 


since the graph for this problem has all the original assignment graph arcs, and 
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has a supply of m at the root node 0 and a demand of one at each of the 
destination nodes m + j, j = 1,..., m. This is depicted in Figure 10.22a, where the 
supplies and demands are shown against each node and the costs are shown on 
the respective arcs. In fact, an optimal basic feasible solution here has basic arcs 
(0, i), i= 1,..., m, and (k; ,m+j),j =1,...,m, where kj € argmin {ey: i= |,..., 


m}. Figure 10.22b shows an optimal basis for the example problem, where the 
optimal dual variables are shown against the nodes. Later, we show that the 
primal feasible problem SP(x) is always dual feasible as well, and hence always 


has an optimal solution. In this case, efficient polynomial-time algorithms 
described in Chapter 12 may be used to obtain a basic optimal solution. Let 
T(X) represent the corresponding optimal basis tree, which will, in fact, be an 


arborescence with node 0 as the root node (e.g., see Figures 10.22b and 10.22d). 
The next step of the algorithm is the flow augmentation step. In particular, the 
basis tree 7(X) obtained at the previous step gives shortest paths from node 0 


through some unassigned origin nodes, to all the unassigned destination nodes. 
Let us pick any such destination node m + (which has a unit demand) and trace 
the chain (reverse path) to the root node 0. This chain must finally pass through 
some unassigned origin node i, say, before reaching the root node 0. Along the 
corresponding path from i to m + j, let us complement the partial assignment 
values. Note that since the origin and destination nodes alternate on this path, as 
do the zero and one partial assignment values, the origin and destination nodes 


in UO or UD on this path remain in these respective sets. Additionally, we have 


i¢ UO and (m+) €UD, so that the cardinality of UO (and hence, UD) falls by 
one. Observe that, in effect, this amounts to sending a unit of flow along this 
path (from node 0), where the x;—variables on this path go from x; = 0 to Xj = 


1, and the y;—variables on this path also go from yy = 9 to yy = 1, Le., the 


corresponding x,—variables transition from x; = 1 to xj; = 0; hence, the 


u] i 
complementation process. If there is another chain (reverse path) from another 
unassigned destination node to the root node 0 that goes through some other 
origin node in the updated UO set, then this step is repeated with respect to this 
path. This is carried out sequentially until the final updated graph contains no 
such path. Note that any two paths considered sequentially at this step must be 
disjoint since the basis tree has no cycles. If UD (and hence, UO) is empty at 
the end of this step, then as shown later, the current assignment xX is optimal. 
Otherwise, we return to the shortest path step and repeat the procedure. Since 


| UD | drops by at least one at each iteration, we can loop through these two steps 
at most m times, which yields an O(m*) polynomial-time algorithm (assuming 


that the complexity of the shortest path algorithm is O(n? )-see Chapter 12). 
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For the example problem, we can trace a path {0, 3, m+ 1} to destination 
node | in Figure 10.22b at the flow augmentation step. This gives x3; = 1, UO 


= {1, 2}, and UD = {2, 3}. A second path to destination 2 can be traced as {0, 1, 
m + 2}, which yields x}. = 1, UO = {2}, and UD = {3}. No additional disjoint 
paths can be traced, and so currently we have the partial assignment solution x, 
= X3, = 1, and x; = 0 otherwise. 

Repeating the shortest path step, the new SP(X) graph obtained is shown 
in Figure 10.22c. An optimal basis tree (arborescence) 7(X) for this problem is 
shown in Figure 10.22d. At the flow augmentation step, we trace the path {0, 2, 
m+ 1, 3, m+ 3} from node 0 to the unassigned destination node m + 3. Note 
that currently, x5, = 0, %3; = 1 (i.e., ¥3; = 0), and X33 = 0 alternate on this path. 
Complementing these values, we get X), = 1, %3; = 0 (e., 93) = 1), and X33 = 1. 
The value x,» = 1 remains unaffected. Hence, the resulting assignment solution 
IS Xj = Xp) = X33 = 1, and x, = 0 otherwise, with UO = UD = ©. We therefore 
terminate, claiming this solution to be optimal, with the corresponding optimal 
dual solution to the assignment problem being given by the optimal dual 
solution to the final shortest path problem SP(x) solved, as shown against the 


nodes in Figure 10.22d. 

To justify this algorithm, consider an optimal basis tree 7(x) for the 
shortest path problem SP(x) at some iteration. Note that, as shown above, an 
optimal solution exists for SP(X) at the initial step when x = 0. Presently, 
assume that an optimum exists for the current shortest path problem SP(x) 


corresponding to the given partial assignment X, and we shall inductively verify 
later than an optimum must then exist for the subsequent shortest path problem 
in case there remain unassigned origins and destinations. Accordingly, let 
W = (W515 Wins Watt o-+> Wm) be the associated set of optimal dual multipliers 


for SP(x). Note that if x, Xj = 0, then arc (i, m + /) is present in the problem 


SP(x) and we have w, — w, with this constraint holding as an equality 


m+ j Sis 
if the arc (7, m +7) belongs to T(x). On the other hand, if x = 1, then arc (m + j, 
i) is present in the problem SP(x) with a cost Cy and, moreover, this arc must 
belong to the shortest path tree 7(X) since this is the only arc coming into node i 


< UO. Therefore, we have Wt j — Wj =—Cy OT Wj — Wy 4 ; = Cy. Consequently, 


+J i] 

w is dual feasible to the assignment problem: Minimize {ex : Ax = b, x > 0} 
and is complementary slack with respect to x (why?). Moreover, the tree T(x) 
also contains the arcs (0, ’), V i € UO, because these are the only arcs coming 
into the respective unassigned origin nodes. Hence, we also have w, = 0, Vie 
UO. 
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Figure 10.23. Example of a transshipment problem. 


Now, let x"°™ be the revised partial assignment solution obtained from ¥ 


based on the flow augmentation step. Consider any (i, /) for which x" = 1. If 


x, = 1 also, then the arc (m + j, i) belongs to T(x) as indicated above and we 
have Wy; —W) =—Cy, L€., Wj —Wm4; =¢j- On the other hand, if x, = 0, then 


the arc (i, m + /) must have been present in 7(X) to be involved in the flow 


augmentation step (whence we obtained x" = 1), and so we again have 


—-W 


m+ j = Cj. Consequently, x"°™ 


Ww; is also complementary slack with respect 


to W. Hence, if x" is a feasible assignment solution, that is, the revised UD = 


©, then it is optimal. Otherwise, we construct the problem SP(x"™ ). But for 


SP(x"™” ), the solution W is still dual feasible, because (a) for the z;—arcs, i € 
UO, we have Wo —w; =0 since Wy) =0, and w,; =0, V i € UO from above; (b) 


for the x,~ares we have W;—-Wy,; S¢j; since W is dual feasible to the 


assignment problem; and (c) for the y,—arcs (corresponding to x;“" = 1), we 


have Wy; —W; =—Cy, L.€., W; —Wy4; = Cy since x""™ is complementary slack 


with respect to W from above. Hence, SP(x"*”) is primal and dual feasible; 


therefore, it has an optimal solution. The algorithmic process can now continue, 
and after at most m iterations, we achieve optimality. 


10.10 THE TRANSSHIPMENT PROBLEM 


In the transportation problem studied in this chapter we have assumed that each 
point is either an origin, where goods are available, or a destination, where goods 
are required. Suppose that, in addition, there are intermediate points where 
goods are neither available nor required, but where goods can be transshipped. 
The problem of finding a shipping pattern having the least cost is called the 
transshipment problem and is illustrated in Figure 10.23. 

This problem is a particular type of network flow problem that can be 
solved directly using the network simplex algorithm of Chapter 9, for example. 
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It is also possible to convert this problem into a transportation problem by 
tracing shortest paths between each origin—destination pair (see Exercise 9.27). 

Another procedure for converting the model into a transportation problem 
is to add buffer stocks at certain nodes, as discussed in Exercise 10.51. 


EXERCISES 


[10.1] Consider the following transportation problem: 


1 2 3. OS; 


Cy matrix 


2) 2/31 442 


a. Construct a basic feasible solution by the northwest corner method. 
Show the basis tree. 
b. Find an optimal solution. 


[10.2] Solve the following assignment problem by the transportation method: 


PERSON 


[10.3] Consider the transportation problem corresponding to the following 
tableau: 


2| 7] 12] 5 | 6 | 30 ¢ matrix 


d, 17 38 20 30 


a. Solve the problem by the transportation algorithm. 
b. Suppose that cj, is replaced by 2. Without resolving the problem, 


find a new optimal solution. 
c. How large should cj), be made before some optimality condition of 


the solution in Part (a) is violated? 
[10.4] The following is a transportation tableau: 
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Sj 
“ 18 

4 
xij 

6 

12 


Is the solution basic? 
Show that the solution is optimal. 
Does this problem have alternative optimal solutions? 
Give the original linear programming problem and its dual. 
Derive the optimal solution to the dual problem. 
Give the optimal simplex tableau associated with the foregoing 
transportation tableau. 

g. Suppose that c43 is increased from 11 to 15. Is the solution still 

optimal? If not, find a new optimal solution. 
[10.5] Consider Exercise 10.4. 
a. Add 10 to each cj. Applying the cycle method, is the tableau still 


meaoge 


optimal? 

b. Multiply each cj by 10. Applying the cycle method, is the tableau 
still optimal? 

c. Repeat Parts (a) and (b) for a general constant k. 

d. What can we conclude from Part (c) for transportation problems? 


[10.6] Solve the following transportation problem: 


Destination 
2 


[10.7] Consider the following transportation problem: 
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Destination 


Cy matrix 


a. Prove by duality theory that 
(1 1512 +%135%215%225%23) = (0, 3, 0, 1, 0, 4) 
is an optimal solution. 
b. Interpret, economically, the dual variables for the solution in Part (a). 


[10.8] Consider the following transportation problem: 


2 a hs a [eas 
d, 10 15 25 15 


a. Give the northwest corner starting basic feasible solution. 
b. Find an optimal solution. 
c. What is the effect on the optimal solution if c;; is changed to 0? 


[10.9] Consider the data of Exercise 10.8. Apply the method of Section 10.1 to 
obtain a feasible solution. Convert the feasible solution into a basic feasible 
solution. Show the basis tree. 


[10.10] Consider the following data for a transportation problem: 


1 2 3 4 8; 
14 | 87 | 48 | 27 | 71 


§2 | 35 | 21 | 81 | 47 cj matrix 


99 | 20 | 71 | 63 | 95 


3 
d, 71 35 47 60 


a. Indicate a starting basic feasible solution by the northwest corner 
rule. Give the basis tree. 
b. Find an optimal solution. 
c. Give the simplex tableau associated with the basic feasible solution 
in Part (a). 
[10.11] The following tableau depicts a feasible solution to a transportation 
problem. Is this solution basic? If not, then starting with this solution, construct 
a basic feasible solution. Compare the costs of the two solutions. Do the columns 
of variables x11, X12, X23, X32, X33, X34, and x, yield a basis? 
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d; 20 10 40 30 


[10.12] On Tuesday, the GT Railroad Company will have four locomotives at 
IE Junction, one locomotive at Centerville, and two locomotives at Wayover 
City. Student trains each requiring one locomotive will be at A-Station, Fine 
Place, Goodville, and Somewhere Street. The local map gives the following 
distances: 


A- Fine Somewhere 
Station Place Goodville Street 
IE Junction 15 38 45 1] 
Centerville 6 61 18 30 
Wayover City 17 14 6 10 


How should they assign power (locomotives) so that the total distance traveled 
is minimized? 

[10.13} An automobile manufacturer has assembly plants located in the 
Northwest, Midwest, and Southeast. The cars are assembled and sent to major 
markets in the Southwest, West, East, and Northeast. The appropriate distance 
matrix, availabilities, and demands are given by the following chart. 


Southwest East West Northeast Sj 
Northwest 1200 8500 1850 2250 2,500,000 
Midwest 400 800 900 1400 1,800,000 
Southeast 800 1200 1000 1100 1,600,000 
d. 2,000,000 1,500,000 1,200,000 1,200,000 


J 


a. Assuming that cost is proportional to distance, find an optimal ship- 
ment pattern. 

b. Assuming that cost is proportional to the square of distance, find an 
optimal shipment pattern. 


[10.14] A company has contracted for five jobs. These jobs can be performed in 
six of its manufacturing plants. Because of the size of the jobs, it is not feasible 
to assign more than one job to a particular manufacturing facility. Also, the 
second job cannot be assigned to the third manufacturing plant. The cost 
estimates, in thousands of dollars, of performing the jobs in the different 
manufacturing plants, are summarized below. 
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PLANT 
JOB 1 2 3 4 2) 6 
1 60 55 42 57 20 52 
2 66 73 — 68 75 63 
3 81 78 72 80 85 78 
4 30 42 38 50 46 42 
5 50 55 40 60 56 70 


a. Formulate the problem of assigning the jobs to the plants so that the 
total cost is minimized. 

b. Solve the problem by the transportation algorithm. 

c. Solve the problem by the Hungarian assignment algorithm. 

d. Apply the primal—dual algorithm to this problem. Make all possible 
simplifications. 


[10.15] An airline company can buy gasoline from three suppliers. The 
suppliers have available 2K, 6K, and 6K gallons, respectively. The company 
needs gasoline at three locations with each location requiring 5K, 3K, and 2K 
gallons, respectively. The per/K gallon quoted price for gas delivered to each 
location is as follows: 


Location 


Suppliers 


How can the company buy the gasoline to minimize the total cost? 


[10.16] Describe in detail how any finite-valued, capacitated minimum-—cost 
network flow programming problem can be transformed into an equivalent 
assignment problem. 


[10.17] Referring to the transportation problem defined in Section 10.1, state 
whether the following are true or false and provide a brief justification. 


a. Ifthe total supply of a proper subset of the origin nodes equals the 
total demand of a proper subset of the destination nodes, then there 
exists a degenerate basic feasible solution. 

b. The entries in any updated simplex tableau column add to unity. 

c. Given an optimal basic feasible solution, if some of the cost coeffi- 
cients of the basic variables are decreased, the solution remains 
optimal. 

[10.18] Devise a method for applying the lexicographic simplex method 
directly on the transportation tableau. Show how the northwest corner rule can 
be used to obtain a starting basic feasible solution with lexicographically 


positive rows in (B'b, B!). 
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[10.19] Show that if the cost coefficient for the artificial variable is increased 
by an amount @ in the transportation problem, then all of the u; and v; variables 


will change by the same amount 6. Show that the cost coefficient of the artificial 
variable does not matter in the computation of Ly — Sy 


[10.20] Show how a transportation problem having rational supplies, demands, 
and cost coefficients can be scaled into an equivalent transportation problem 
having integer data. 


[10.21] Show that if we define 


S=s +é, t=1,..,m 
ad;= -P J=1, nl 
d, =d,+me, 


then by a proper choice of € we can totally avoid degeneracy in the transporta- 
tion problem (and maintain an equivalent problem). 


[10.22] Consider a transportation problem having m sources and n sinks with 
respective supplies and demands s;, i= 1,..., m, and d;, j= 1,..., n. Assume that 


only a subset .~/ of the possible arcs (i, j) are present in the problem having 
given cost coefficients ¢, and with each source and sink node having at least 
one incident arc. Create a dummy source node (m + 1) with supply equal to 
Liat d;, and create a dummy sink node ( + 1) with demand equal to L7", 5;. 
Also, construct slack arcs (i, 2 + 1), i = 1,..., m+ 1 with zero cost coefficients, 


and construct artificial arcs (m+ 1, j), j = 1,..., 2, with cost coefficients equal to 
M. 


a. Show that the following value of M is sufficient to ensure that the 
total flow on the artificial arcs is at its minimum feasible value at 
optimality. 


M-= 1 + {sum of the @ largest cj —values, (i,j) €-/} 


~— {sum of the (Q — 1) smallest cj values, (i,j) €.°/ } 


where Q = minimum {m, n,| |. Y|/2}, and where | -] denotes the 


rounded—up integer value. 
b. Examining the role played by M in your proof, comment on how 
alternative bounds on M may be derived. 


[10.23] Consider the following problem: 
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Minimize 3: x CyXiy 
i=l j=l 
‘ n 
subject to > Xy = Sis i=l,...,m 
j=l 
m * 
LX PyXy = 4; J=i\,...n 
i=l 
Xj 20, i=1,...,.m, j=1,...,n, 


where pj > 0 for all i, j. Extend the transportation algorithm of this chapter to 


handle the foregoing problem (which is sometimes referred to as the generalized 
transportation problem). 


[10.24] 


Formulate the problem of Exercise 1.13 as a generalized transportation 


model, and use the procedure of Exercise 10.23 to find an optimal solution. 


[10.25] 


The following is Vogel’s approximation method for obtaining a 


reasonably good starting basic feasible solution to a transportation problem: 


Step 0. 
Step 1. 


Step 2. 


Step 3. 


Step 4. 


Step 5. 


[10.26] 


Begin with all cells unallocated. 

In the problem at hand, compute for each row and each column the 
difference between the lowest and next lowest cost cell in the row or 
column. 

Among those rows and columns at hand, select the one that yields a 
maximum difference. 

Allocate as much as possible to the x,;—variable having the least cost 


coefficient in the selected row or column. Decrease the corresponding 
supply and demand. Drop the row or column whose supply or demand 
is Zero. 

Make any allocations where only one unallocated cell remains in a row 
or column. After reducing the corresponding supply and demand and 
dropping the row or column, repeat Step 4 as necessary. 

Stop if no rows and columns remain. Otherwise, return to Step 1 with 
the reduced problem. 


a. Show that Vogel’s approximation method leads to a basic feasible 
solution, after including any required zero cells. 

b. Apply Vogel’s method to the data of Exercises 10.2, 10.6, and 
10.8. 


The following is the matrix minimum method for obtaining a reasonably 


good starting feasible solution for a transportation problem: 


Step 0. 
Step 1. 


Step 2. 


Begin with all cells unallocated. 

Identify the lowest-cost unallocated cell in the matrix and allocate as 
much as possible to this cell. 

Reduce the corresponding supply and demand, dropping the one going 
to zero, and repeat Step | until all supplies and demands are allocated. 


a. Show that the procedure produces a basic feasible solution. 
b. Apply the procedure to the data of Exercises 10.6 and 10.8. 
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[10.27] Show how sensitivity analysis with respect to varying each of the c;-, 


s;—, and d parameters can be conducted on a transportation tableau. 


[10.28] Prove or give a counterexample: For a variable to be basic in a particu- 
lar row of the linear program for a transportation problem, the variable must 
have a nonzero entry in that row. 


[10.29] Show how the dual simplex method can be applied to transportation 
problems directly on the transportation tableau. 


[10.30] Devise a formal procedure for identifying the unique cycle of basic 
cells associated with an entering nonbasic cell. The procedure must specify the 
cells having coefficients 1, —1, and 0 in the representation of the nonbasic cell. 
(Such a procedure must be developed when the transportation algorithm is 
coded on a digital computer.) 


[10.31] Give the dual of a transportation problem. Is it possible to readily 
specify a feasible solution to the dual problem? If so, starting with this solution, 
devise a method for applying the primal—dual method directly to a transportation 
tableau. 


[10.32] Consider the following capacitated transportation problem: 


ahs mn 
Minimize Ds CX 


i=1 j=l 
n 
subject to XY Xyj = Sj i=1,...,m 
j=l 
m 
XL xy = 4), J=1, of 
i=l 
Osx; Sujy, i=],...,m, J=l,.u.n. 


Specialize the bounded variables simplex method of Chapter 5 (or Chapter 9) to 
solve the foregoing transportation problem on the transportation tableau. 
Describe all details: Finding a starting basic feasible solution, computing the dual 


variables and the (z,—cj )—values, and determining the entering, and the 


leaving variables. 
[10.33] Consider Exercise 1.14. 
a. Formulate the problem. 
b. Solve the problem by the capacitated transportation algorithm that you 
developed in Exercise 10.32. 
c. Suppose that the third manufacturing company lost one of its other 
contracts so that 700 tons are available to the furniture company. Find a 
revised optimal solution. 


[10.34] Apply the Hungarian method to the following assignment problem: 


Cost matrix 
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Os Ae [26 |o Sel S| 
5 Lo? 4]5[2[7] 


[10.35] Given an optimal reduced matrix for an assignment problem, show how 
to construct an optimal basis tree for the associated linear program. Demonstrate 
by the following reduced matrix: 


{10.36] A carpenter, plumber, and engineer are available to perform certain 
tasks. Each person can perform only one task in the allotted time. There are four 
tasks available, three of which must be done. The inefficiency matrix for person 
i assigned to task 7 is as follows: 


SOLDERING _ FRAMING DRAFTING _ WIRING 


Carpenter 4 6 4 4 
Plumber 3 4 2 3 
Engineer 7 5 6 5 


Which person should be assigned to which job? (Hint: Create a dummy person.) 
Which job will go unfinished? Now suppose that each person can perform up to 
two tasks, and all tasks must be done. What should they do? 


{10.37] Sally, Susan, and Sandra will go on a date with Bob, Bill, and Ben. 
Sally likes Bill twice as much as Bob and three times as much as Ben. Susan 
likes Bill three times as much as Bob and seven times as much as Ben (Ben is a 
loser!). Sandra likes Bob about as much as Bill, but likes them both about six 
times as much as Ben. How should the couples pair up so that in the aggregate 
the girls are as happy as possible? If one girl is willing to stay home, which one 
should it be? Which boy will lose out? (You guessed it!) 


{10.38] Describe a procedure suitable for computer coding that will find, 
directly on the reduced matrix, the maximum number of independent zero cells 
in the reduced matrix (or equivalently, the minimum number of lines to cover all 
zeros). The reader may wish to study Exercise 12.17. 


[10.39] Use the theorems of duality to prove that the minimum number of lines 
to cover all zeros in a reduced assignment matrix equals the maximum number 
of independent zero cells. (Hint: Consider the problem: 


Maximize LLxy 
subject to by a,x; <1, 
x; 20, VG,/), 


where a; =e; +e and the sum in the constraint is taken only over zero cells - 


ij 
of the reduced matrix. Take the dual of this linear program and examine its 
properties.) 


m+ ij 
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[10.40] Compare the Hungarian method for the assignment problem with the 
primal—dual method applied to the assignment problem. 


[10.41] Show that every vertex of the assignment polytope of Section 10.7 has 
(2”!)(m™~*) bases representing it. 


[10.42] Referring to the assignment polytope of Section 10.7, show that two 
extreme points x # y are adjacent if and only if P(x)U P(y) contains exactly 


one cycle, where P(x), for a given vertex x, is the subgraph of the assignment 


graph that contains only the positive flow arcs (see Figure 10.20a, for example). 
Show that given any pair of extreme points x # y of the assignment polytope, 


either x and y are adjacent or there exists an extreme point z such that x and z as 
well as y and z are adjacent. (Hint: Recall that x and y are adjacent if and only if 
they have bases representing them such that the union of the respective spanning 
tree graphs contains exactly one cycle.) 


[10.43] Show that each new dual solution in the assignment procedure specified 
in Section 10.7 is feasible. 


[10.44] Show how one can construct the simplex tableau associated with an 
optimal assignment matrix. 


[10.45] a. Show that a feasible basis for an assignment problem is an 
alternating path basis if and only if all one—arcs are pointing away 
from the root, which is also equivalent to having an origin node as 
the root node, with a single arc incident at the root node, and with 
all other origin nodes having exactly two incident arcs. 

b. Give equivalent definitions of “alternating path bases” by designat- 
ing a demand node as the root node. 


[10.46] Consider an assignment problem for which the underlying bipartite 
graph is not complete. Suppose that we construct the shortest path problem 
SP(x) for some partial solution x, as in the successive shortest path algorithm of 


Section 10.9. Show that if SP(x) is infeasible, then so is the original assignment 


problem. (Hint: Show that based on a feasible assignment, if one exists, a (reverse) 
path can be constructed from any unassigned destination node to the root node 0 
in SP(X).) 


[10.47] Show that given any basis representing an extreme point of the 
assignment polytope, the maximum number of potential nondegenerate pivots 
on this basis graph is m(m — 1)/2, and the minimum number is (m ~ 1). Show 
that the maximum number is achieved if and only if the basis tree is a chain 
graph (see Figure 10.20b) and the minimum number is achieved on a basis in 
which all the degenerate arcs originate at some node or all terminate at some 
node. Can both types of bases be alternating path bases? 

[10.48] Is it true that a feasible basis for an assignment problem in which some 
origin node is the root node, and in which the one—arcs and the zero—arcs alter- 
nate on the chain from any node to the root node is an alternating path basis? 
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[10.49] Solve the problem in Exercise 10.2 using the successive shortest path 
algorithm of Section 10.9. At each iteration, give the basic primal and dual 
optimal solutions to the shortest path problem SP(x). Demonstrate that this 
dual solution is feasible to the original assignment problem as well as to the 
updated shortest path problem SP(x,¢y,), and that it is complementary slack 


with respect to both X and X,.y. 


[10.50] Using the alternating path basis algorithm for solving assignment prob- 
lems, provide details for all the specializations you can make in the algorithmic 
operations, including any specializations in the list structures of Chapter 9. 
[10.51] Given a transshipment problem, the following procedure is suggested to 
convert it into a transportation problem. First the nodes are classified into the 
following mutually exclusive categories: 


1. Pure source: a node that only ships. 
2. Pure sink: a node that only receives. 
3. Transshipment node: a node that may ship and receive. 


A transportation tableau is constructed as follows. The origins are the pure 
sources and the transshipment nodes. The availability at each transshipment 
node i is replaced by s; + B, where s; is the maximum of zero and the net out of 


node i, and B is a buffer stock to be specified later. The destinations are the pure 
sinks and the transshipment nodes. The requirement at a transshipment node i is 
d; +B, where d; is the maximum of 0 and the net into node i. If there is no 


direct link from node i to node /, then Cy is equal to M, where M is a large 
positive number. Also, c;; = 0 for transshipment nodes. Finally, B is a large 


positive number, say, B = >'s;. 
i 


a. Using the foregoing instructions, form the transportation tableau 
corresponding to the above transshipment problem, where the 
availability at nodes 1, 2, and 3, are respectively, 10, 20, and 15, and 
the requirements at nodes 5, 6, and 7 are, respectively, 10, 25, and 10. 

b. Solve the problem using the transportation algorithm. Interpret the 
solution. What is the interpretation of the buffer B? 

c. Show that the procedure outlined in this exercise is valid in general. 
(Hint: On the original flow conservation constraints, use appropriate 
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substitutions of the type where a new variable equals B minus the 
sum of some variables.) 

d. Convert the problem into a transportation problem using the least 
cost method discussed in Section 10.10. Apply the transportation 
algorithm and interpret your solution. 

{10.52] If it is known in advance that a certain variable will be positive in any 
optimal solution to a transportation problem, what simplifications can result in 
the solution method? 


mm 
[10.53] Consider the assignment problem AP to minimize + > cyXy» Subject 
i=l j=l 


m m 
to 2 x; =1 for each origin i = l,...,m; ->Y x =—I1 for each destination j = 
jal i=l 


1,...,m, and x > 0. Suppose that we are applying the successive shortest path 
algorithm to solve this problem, where at a certain main step in this procedure, 


we are given a partial assignment x having |UO|=|UD|=q 21 and |vo| = 


| UD|=m-q>1. Accordingly, construct a relaxation of Problem AP as 


follows: 
(i) Aggregate the unassigned origin constraints to obtain 
m m 
x > xy = q. Now, define z; = > x;, Vi € UO, and 
icUO j=l jal 
rewrite this equation as the following set of constraints: 


ij? 


xX 4=49 
icUO 


m 
> xy — 2 =9, Vie UO. 
j=l 


(ii) For each (i, j) such that x; = 1, use a change of variables y;; = 


] — xj, and impose yj 2 0 (but not Vy FN, 1e., relax xj; 2 0 for 


these x;;—variables). 


a. Show that the resulting relaxed problem is a minimum cost network 
flow program that is defined as SP(X) in Section 10.9. 


b. Explain how the flow augmentation step of the successive shortest 
path algorithm ensures that the relaxed nonnegativity constraints in 
the transformation (ii) above are not violated when revising X to 


yhew 
x . 
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NOTES AND REFERENCES 


1. Hitchcock [1941] is credited with the first formulation and discussion of 
a transportation model. Dantzig [1951c] adapted his simplex method to 
solve transportation problems. Charnes and Cooper [1954] developed an 
intuitive presentation of Dantzig’s procedure through what is called the 
“stepping stone” method. In this chapter, we called this the “cycle 
method.” 

2. Koopmans [1949] was the first to note the relationship between basic 
solutions in the transportation problem and the tree structure of a graph. 
Other good discussions are provided by Dantzig [1963a] and Johnson 
[1965]. 

3. The bound on the value of M in Exercise 10.22 is derived in Sherali 
[1988]. 

4. A good study of the properties of the assignment polytope appears in 
Balinski and Russakoff [1974]. In addition to the results in Exercises 
10.41 and 10.42, they show that the assignment polytope is of dimension 


(m — 1% has m® facets, and has a simplex path linking any two vertices 


of length no more than 2m — 1. Therefore, the Hirsch conjecture (see the 
Notes and References section of Chapter 8) holds for this polytope. 

5. The Hungarian method for the assignment problem was developed by 
Kuhn [1955]. This method finds its roots in the work of the Hungarian 


mathematician Egervary [1931], hence its name. Kuhn’s paper led to the 


general primal—dual method for linear programs in the following year. 

6. McGinnis [1983] presents a computational comparison between the 
Hungarian algorithm and the primal simplex alternating path basis algo- 
rithm. For improvements and variations in the Hungarian algorithm, see 
Bertsekas [1981], Carpenito and Toth [1980], Redlack and Huang [1987], 
Jonker and Volgenant [1986], and Sherali and Driscoll [2003]. In 
particular, the last two papers examine alternatives to simply covering the 
zeros in the reduced matrix using the fewest number of lines versus 
focusing on a maximal dual improvement. Balinski [1984, 1985] 
provides an excellent study of the dual polyhedra of transportation and 
assignment problems, and exhibits why dual—based simplex approaches 
may be more suitable for these problems than primal procedures. 

7. The alternating path basis algorithm was developed by Barr et al. [1977]. 
This algorithm turns out to be a special case of the strongly feasible tree 
algorithm of Cunningham [1976] that was independently developed for 
general network flow problems. The results in Exercises 10.45 and 10.50 
are from the former paper. The result in Exercise 10.47 is from Bazaraa 
and Sherali [1982]. 

8. The successive shortest path algorithm for assignment problems dis- 
cussed in Section 10.9 is from Glover et al. [1986]. The threshold shortest 
path subroutine is used to derive a computationally very efficient 
algorithm. For another successive shortest path algorithm, see Engquist 
[1982]. Other polynomial algorithms for the assignment problem appear 
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in Akgul [1986a], Balinski [1985] (see Goldfarb [1985] for an 
improvement of this “signature algorithm’), Balinski [1986], and Hung 
[1983]. Also, see Orlin [1985] for some relevant discussions. 

9. The reader interested in a further study of transportation problems 
involving aggregation and disaggregation methods for large-scale prob- 
lems is referred to Balas [1965], Zipkin [1980], and Zipkin and Raimer 
[1983]. 
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ELEVEN: THE OUT-—OF-— 
KILTER ALGORITHM 


In Chapter 9, we presented a network simplex method for solving minimal—cost 
network flow problems. In this chapter, we present another method for solving 
minimal-—cost network flow problems, called the out—of—kilter algorithm. This 
algorithm is similar to the primal—dual algorithm in that it begins with dual 
feasibility, but not necessarily primal feasibility, and iterates between primal and 
dual problems until optimality is achieved. However, it differs from the primal— 
dual algorithm (as strictly interpreted) in that the out—of—kilter algorithm does 
not always maintain complementary slackness. In fact, the principal thrust is to 
attain complementary slackness. A version of the algorithm can be designed in 
which we maintain primal and dual feasibility (not necessarily basic solutions) 
and strive to achieve complementary slackness. Computationally, the state-of— 
the-art primal simplex codes run two-three times faster than traditional out-of- 
kilter codes. However, advances in primal—dual methods that use the basic 
ingredients of the out-of-kilter algorithm, although in a manner different from 
this algorithm, have resulted in algorithms that run two—four times faster than 
the best primal simplex code with the speed-up factor increasing by an order of 
magnitude for large problems. (We provide some related comments on such 
procedures in the concluding section of this chapter, and refer the reader to the 
Notes and References section for further reading on this subject.) From this 
viewpoint, the concepts of the various algorithmic steps presented in this chapter 
are important. 


11.1 THE OUT—OF-KILTER FORMULATION OF A MINIMAL-COST 
NETWORK FLOW PROBLEM 


For convenience of presentation, the form of the minimal—cost flow problem 
that we shall work with is: 


* * * m m 
Minimize > > Cy Xi 


i=l j=l 
7 m m 
subjectto Yi xy — Y x4 =0, i=l,...,m (11.1) 
ji k=l 
xy 2 ly, i,j =1,....m 
Xi Suy, i,j =1,...,m, 


where it is understood that the sums and bounding inequalities are taken over 
existing arcs only. We call any flow (choice of the Xi —variables) that satisfy the 


equality constraints in Problem (11.1) a conserving flow. A conserving flow that 


satisfies the remaining constraints £;; < xj < uy is a feasible flow (solution). 
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We shall assume that Cis é,., and uj are integers and that —co < / ip S Uy <2. 


y? y 
Since all right-hand-side values of the flow conservation equations in Problem 
(11.1) are zero, we conclude that the flow in the network does not have a 
starting point or an ending point, but circulates continuously throughout the 
network. Thus a conserving flow in the network will involve flows along cir- 
cuits (directed cycles). For this reason, the representation in Problem (11.1) is 
known as a circulatory network flow problem. 

The foregoing formulation is completely equivalent to the formulation of 
the minimal—cost network flow problem presented in Chapter 9. This is readily 
seen by noting that a minimum cost network flow problem can be first of all put 
in the form (11.1) with a general (integer) right-hand-side vector b instead of a 
zero vector in the flow conservation constraints, and in particular, with 0 < ¢ es 


uj < © for all (i, j). Here, we assume that the variables xij that are unbounded 


from above are artificially bounded by using uy = M, where M is sufficiently 
large. By replacing each right-hand-side magnitude |b, | with a variable y,, where 
y; is bound restricted as |b;| < y; <|B,|, it is readily seen how we can transform 


the given problem into the form (11.1) (see Exercise 11.1). Hence, in particular, 
note that we may possibly have some £; = uj; in Problem (11.1). 


We emphasize here that the homogeneous form in Problem (11.1) is only 
for the sake of convenience. The same algorithm discussed in the sequel is 
applicable with a general right-hand-side vector and with -c0 < £;; < uy < ©, 


given a starting solution that is flow—conserving (perhaps to an artificial net- 
work). More specifically, consider a bounded variables network flow problem in 
the following general form as discussed in Chapter 9. 

Minimize {cy : Ay =b, £< y <0}, 


where A is a node~arc incidence matrix, and where —o < by < ui <oo, for all 
arcs (i, j). Now let y be any arbitrary flow—conserving solution that satisfies 
Ay =b, and consider the transformation 

x=y-y, 1.e., y=yt+x. 
Under this transformation, the preceding problem becomes: 


Minimize {c(y + x): Ay + Ax =b, €-y <x <u-y}. 
Dropping the constant term cy from the objective function, using the fact that 


Ay =b, and defining £= @—y and u=u-y, this problem is equivalent to: 
Minimize {ex : Ax = 0,2 < x <u}, 


where —20 < ij Uy < ©. This problem is now in the form (11.1). Note that in 


Mi 
this form, x can be interpreted as the modification in the given flow y that is 
required to achieve an optimal solution (if one exists). Hence, Problem (11.1) 


can essentially be viewed as a flow augmentation formulation of a minimal—cost 
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network flow program. We ask the reader to explore this concept further in 
Exercises 11.2 and 11.3. 


The Dual of the Circulatory Network Flow Problem and Its Properties 


If we associate a dual variable w; with each node’s flow conservation equation 
S uy (which is 


in Problem (11.1), a dual variable hi with the constraint Xy 


treated as SK 


with the constraint red £.. 


2 —uj for the purpose of taking the dual), and a dual variable v,; 


re the dual of the out—of-kilter formulation for the 


minimal—cost network flow problem is given by: 


wf mim mim 
Maximize YY fjvy- 2 fe ghy 
i=l j=l 


subject tow, Ww, ti 2h =e i,j =),....m 
hy Vi = 0, i,j =1,....m 
w; unrestricted, i=1,...,m 


qt 
where the summations and the constraints are taken over existing arcs. The dual 
problem has a very interesting structure. Suppose that we select any set of w;— 


values (we shall assume throughout the development that the w;-—values are 


integers). Then the dual constraint for arc (i, /) becomes 
Vig iG = Cy TWiT Wy hy 2 0, Viz 
and is satisfied by letting 


Vij = maximum {0,¢j —W; +w; } 


hy = maximum {0,—(cj — w; + w;)}. 


20, 


Thus, the dual problem always possesses a feasible solution given any set 
of w;—values. In fact, the choices of Vi and hy just given yield optimal values 


of v, and hy for a fixed set of w; —values (why?). 


The Complementary Slackness Conditions 


The complementary slackness conditions for optimality of the out—of-kilter 
formulation are (review the Karush-Kuhn—Tucker optimality conditions) the 
following: 


(xy — 2 )vy =9, i,j=1,..,m (11.2) 
(uy — xy hy =9, ij =1,...,m. (11.3) 
Define zj —cj = w;—w ; —cj. Then by the definition of vj; and hy we get 
Vy = maximum {0,-(z, -— oy )} (11.4) 
hy = maximum {0, 2; — cj}. (11.5) 


Note that zj—cj; would be the familiar coefficient of x, in the objective 


function row of the lower—upper bounded simplex tableau if we had a basic 
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solution to the primal problem. However, we need not have a basic solution 
here, and no such implication of a basis is being made here by this notation. 


Given a set of w;—values, we can compute 2 —cy = w;— Ww; -cy. 


Noting Equations (11.4) and (11.5), the complementary slackness conditions 
(11.2) and (11.3) hold only if 


Zz ey <O> vj >O0> xy = 4,5 ij=l,...,m 


i 


Zi, — Ci >0>h; >O0> xy = Uy, 


Conversely, if xj = ¢,; for all (i, 7) such that z—cj <0, and if x; =u 


i,j = 1,..., mM. 


ij 
for all (i, ) such that z;; —cj > 0, then defining v,; and h, as in Equations (11.4) 
and (11.5), respectively, we have complementary slackness holding true. Hence, 


we obtain the following key result that embodies the optimality conditions of 
Problem (11.1), as well as the principal thrust of the out—of-kilter algorithm. 


Theorem 11.1 


Let x be any conserving flow, and let w = (,...,W,,) be any integer vector. 


Then x and w are, respectively, primal and dual optimal solutions to Problem 
(11.1) if and only if for all (i, /) 

2 —¢jy <O implies x; = 2;,, 
2 —¢y > Oimplies xy = Uy 

zy —¢y =Oimplies €,, < xy Suy, 

where (2 — cy) = w,—w, —¢y for all (i, j). 


The problem then is to search over values of the w;—variables and flow 


conserving x, variables until the three conditions of Theorem 11.1! are 


satisfied. 
Consider Figure 11.1a. Selecting a set of starting w,;—values, say, each 


w; = 0, and a conserving flow, say, each x; = 0, we can check for optimality. 
Figure 11.1b displays the values for zj;—cj, xj, and w; for the network of 


Figure 11.1a. In Figure 11.1b we see that z), —cjy =—2 and x42 =0 (= 4)9), and 
thus are (1, 2) is said to be in-kilter, that is, well. On the other hand, 
293 —€73 =3 and X73 =0 (<w3), and thus arc (2, 3) is said to be out-of-kilter, 
that is, unwell or in improper condition. Hence, the name out—of-kilter. 

To bring arc (2,3) into kilter we must either increase x23 or decrease 
293 —Cz3 by changing the w;—values. This is exactly what the out—of-kilter 
algorithm attempts to do. During the primal phase of the out—of-kilter algorithm 
we shall be changing the x,—values in an attempt to bring arcs into kilter. 
During the dual phase we change the w,—values in an attempt to reach an in- 
kilter state. 
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The Kilter States and Kilter Numbers for an Arc 


The in-kilter and out—of-kilter states for each arc in a network are given in 


Figure 11.2. Note that an arc is in kilter if £;; <x Suy and the conditions (of 


Theorem 11.1) hold true. As we change the flow on arc (i, /), the arc moves up 


and down a particular column in Figure 11.2 depending on whether xy is 


increased or is decreased. As we change the w; —values the arc moves back and 


forth along a row. Figure 11.2b gives a graphical depiction of the kilter states of 
an arc. Each of the cells in the matrix in Figure | 1.2a corresponds to a particular 
subregion in Figure 11.2b. 

In order to assure that the algorithm will converge, we need some meas- 
ure of the “distance” from optimality. If we can construct an algorithm that 
periodically (at finite intervals) reduces the distance from optimality by an 
integer, then the algorithm will eventually converge. 

There are many different measures of distance for the out—of-kilter 
method. We present in Figure 11.3 one measure of distance that we call the 
kilter number Kj for an arc (i, j). The kilter number is defined here to be the 


minimal change of flow on the arc that is needed to bring it into kilter. The 
kilter number of an arc is illustrated graphically in Figure 11.3b. Notice that 
since all terms involve absolute values, the kilter number for an arc is nonnega- 
tive. Also, notice that if the arc is in—kilter, the associated kilter number 


(Lj sij Cy) 


(a) (b) 
Figure 11.1. An example network: (a) The network. (b) Values of w;,, Zij - 


Cij» ij: 
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Ze; <0 z.-c,=0 z,-c,>0 


Out-of-kilter 


Out-of-kilter 


Out-of-kilter 


Out-of-kilter 


Out-of-kilter 


Out-of-kilter Out-of-kilter 


In-kilter 


In-kilters=: 


Out-of-kilter 
i 


Out-of-kilter 


(b) 


Figure 11.2. The possible kilter states for an arc. 


is zero, and if the arc is out—of-—kilter, the associated kilter number is strictly 
positive. Note that if 24, —Cj <0, then arc (i, /) is in-kilter only if the flow is 


equal to ¢,,, and hence the kilter number x, -£ il indicates how far the current 


i? ij 


flow xy is from the required value if- Similarly, if Z —¢ 7 9, then the kilter 


number 


Xj —uj| gives the distance from the required flow of uj. Finally, if 


Zi; —C = 0, then the arc is in—kilter if by S xy S uy. In particular, if xy > u 


yj “i ij yp 


u 


then the arc is brought in-kilter if the flow decreases by xy —uj|, and if xy < 


é,,, then the arc is brought in-kilter if the flow increases by Xj —£,), and 


iy ’ 
hence we obtain the entries in Figure 11.3, as shown under the column z,; —cj 
=0. 
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Figure 11.3. The kilter numbers K 


One method of assuring finite convergence of the out-of-kilter algorithm 
is to guarantee the following: 


1. The kilter number of any arc never increases. 
2. At finite intervals, the kilter number of some arc is reduced (by an 
integer). 


This is exactly what we shall be able to achieve. 


11.2 STRATEGY OF THE OUT-OF-KILTER ALGORITHM 


As indicated before, the out—of-kilter algorithm may be generally viewed as a 
primal—dual type of algorithm. In this respect the generic steps of the algorithm 
are as follows: 


1. Begin with a conserving flow, such as each xy = 0, and a feasible 


solution to the dual with w arbitrarily chosen, such as each w, = 0, 
and with hy and vy as defined in Equations (11.4) and (11.5) for ail 


arcs (i, /). Identify the kilter states and compute the kilter numbers. 
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2. Ifthe network has an out—of-kilter arc, conduct a primal phase of the 
algorithm. During this phase an out—of-kilter arc is selected and an 
attempt is made to construct a new conserving flow in such a way 
that the kilter number of no arc is worsened and that of the selected 
arc is improved. 

3. When no such improving flow can be constructed during the primal 
phase, the algorithm constructs a new dual solution in such a way 
that no kilter number is worsened and Step 2 is repeated. 

4. Iterating between Steps 2 and 3, the algorithm eventually constructs 
an optimal solution or determines that no feasible solution exists. 


The Primal Phase: Flow Change 


During the primal phase, the out—of—kilter algorithm attempts to decrease the 
kilter number of an out—of—kilter arc by changing the conserving flows in such a 
way that the kilter number on any other arc is not worsened. Examining Figure 
11.3, we see that the flows must be changed in such a way that the 
corresponding kilter states move closer to the in-kilter states. For example, for 


the out-of-kilter state x; > uj and z;—cj <0, we can decrease x, by as 


much as |x £ il before the arc comes into kilter. If we decrease x, beyond 


i 
this, the arc will pass the in-kilter state (we do not want this to happen). Also, 


we do not permit any increase in this x, value. A similar analysis of the other 


kilter states produces the results in Figure 11.4a. 


Several cells in Figure 11.4a deserve special attention. The out-of—kilter 


state xj > uy and zy a 0 indicates that the flow can be decreased by as 


£,;|. Referring to Figure 11.3, we see that we really only need to 


much as |x; - 


decrease the particular xy by |xy —uy|, a smaller amount, to reach an in-kilter 


state. However, as can be seen in Figure 11.3, we may continue to decrease x;j 


by an amount up to 


xj —£,| from its original value and the arc will still remain 


in—kilter. It might be indeed desirable to do this in order to aid other arcs in 
reaching in-kilter states. Also, an arc in the in-kilter state for which 2; < xj < 


uj and zj;—cy = 0 may have its flow appropriately either increased or 


ij 
decreased, while still maintaining its in—kilter status. Figure |1.4b illustrates the 
permitted flow changes graphically. 


Now that we have ascertained how much an individual flow on an arc 
may change, we must still determine what combination of flows we can change 
in order to maintain a conserving flow. If X is the vector of (current) conserving 
flows, then the conservation of flow equality constraints in Problem (11.1) can 
be rewritten as Ax =0, where A is the node—arc incidence matrix. If A is a 


vector of flow changes, then we must have 
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(b) 
Figure 11.4. Permitted flow change directions and amounts. 


A(x+A)=0 or AA =0. 
If AA=0 for a nonzero A, then the columns of A corresponding to the nonzero 
components of A must be linearly dependent. Since A is a node—arc incidence 
matrix, then each column of A has exactly one +1 and one —1, and the nonzero 
components of A must correspond to a (not necessarily directed) cycle or a set 
of cycles (why?). Hence, flows must be changed along a cycle or a set of cycles 
in order to continue satisfying the conservation of flow equations. 

Given an out—of-kilter arc, we need to construct a cycle containing that 
arc. This cycle must have the property that when assigned an orientation and 
when flow is added, no arc has its kilter number worsened. A convenient 
method for doing this is to construct a new network G’ from the original 
network according to the information in Figure 11.4. First, every node of the 
original network is in the new network. Next, if an arc (i, /) is in the original 
network and the flow may be increased, then arc (i, 7) becomes part of the new 
network with the appropriate permitted flow change being as indicated in Figure 
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11.4. Finally, if an arc (i, /) is in the original network and the flow can be 
decreased, then arc (/, 7) becomes part of the new network with the permitted 
flow change being as indicated for arc (i, /) in Figure 11.4. Arcs in the original 


network having lig < xy < ug and zz —cy = will produce two arcs, (i, /) and 


(j, i), each with an appropriate permitted flow change in the new network. Arcs 
not permitted to change in flow are omitted entirely from G’. 

Given the example indicated in Figure 11.1, a new network G’ is con- 
structed by the foregoing rules and is presented in Figure 11.5. To illustrate, 
consider arc (1, 3) in Figure 11.1. Note that x13 <¢)3 and z,3—c,3 <0. From 
Figure 11.4 the flow on (1, 3) can increase to £;3 = 1. This results in arc (1, 3) 
in Figure 11.5, with a permitted flow change of 1. 

Once the new network G’ is constructed and an out-of-kilter arc (p, q) 
in G’ is selected, we look for a circuit (directed cycle) containing that arc in G’. 
This circuit in G’ corresponds to a cycle in G. The flow in the cycle in G is 
changed according to the orientation provided by the circuit in G'. The amount 
of change is specified by the smallest permitted flow change of any arc that is a 
member of the circuit in G’. If no circuit containing the selected out-of-kilter 
arc exists in G’, then we proceed to the dual phase of the algorithm. 

We remark here that the construction of G’ presented is only for 
pedagogical purposes. In essence, given an actual out-of-kilter arc (p, g) in G’, 
we wish to determine if there exists a circuit in G’ that involves this arc. Note 
that this arc may be (p, q) or (q, p) in G, depending on whether an increase or a 
decrease in the flow on this arc is required in G. Hence, in G’, we need to find a 
(directed) path from q to p. This may be done by constructing a tree T that 
begins with the node gq, and at each stage scans for a pair of nodes i and / with 
ieéT and j¢T such that there is an arc (i, /) in G that can permit a flow 
increase or such that there is an arc (/, i) in G that can permit a flow decrease. 
Arcs of this type are called /abel eligible arcs. If such an arc exists, it is used to 
include node j in the tree T and the process repeats. If node p gets included in T 
at any step, then we will have found a circuit in G’ oriented along (p, q) that 
permits a positive flow change. This situation is called a breakthrough. If no 


Permitted flow 
3 1 6<— change 


4 
5 


Figure 11.5. The modified network G’ for Figure 11. 1. 
additional nodes can be added to T at some step and p ¢T, then there does not 


exist any circuit in G’ involving (p, q) (why?). This situation is called a 
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nonbreakthrough. In this event, we proceed to the dual phase with the nodes in 
T forming a set X and the remaining nodes in X. A labeling scheme that 
constructs such a tree T is described in Section 11.5. For now, let us continue to 
work with the network G’. 

As an illustration of the primal phase, consider the modified network G’ 
of Figure 11.5. We select an out—of-kilter arc, say, (1, 3). From Figure 11.5, we 
see, that a circuit exists in G’ containing arc (1, 3), namely, C = {(1, 3), (3, 4), 
(4, 1)}. Hence, we can change the flow around the associated cycle in G, 
increasing flows on arcs having the orientation of the circuit in G’ and 
decreasing flows on arcs against the orientation of the circuit in G’, and obtain 
an improved (in the kilter number sense) solution. The amount of permitted 
change in flow is A= minimum {1, 5, 3) = 1. The new solution and associated 
modified network is given in Figure 11.6a. Arcs (2, 3) and (3, 4) are still out— 
of-kilter in G. Selecting one of the associated arcs in G’ (see Figure 11.6b), say, 
(2, 3), we attempt to find a circuit in G’ containing the selected arc. Because no 
such circuit exists, we must pass to the dual phase of the out—of-kilter 
algorithm. The tree 7 in this case consists of nodes 3, 4, and 1, and arcs (3, 4) and 
(4, 1) from G’. 

It is convenient (but not necessary—see Section 11.6) for the various 
proofs of convergence to work on the same out—of-kilter arc (p, q) until it 
comes in-kilter. We shall assume throughout our discussion of the algorithm 
that this is done. 


The Dual Phase: Dual Variable Change 


When it is no longer possible to construct a circuit in G’ containing a specific 
out-of-kilter arc, then we must change the (z,; — cj )—values so that no kilter 


number is worsened and either the out—of-kilter arc is brought into kilter or 
some new arcs are introduced into G’ that would eventually allow us to find a 
circuit containing the out—of-kilter arc under consideration. 


—G7 =2 
(212 12 (3) 


X= 0 Permitted flow 


J change 


(a) G (b) G’ 
Figure 11.6. The new solution for the network of Figure 11. 1. 
Since z; —cy =w;—w,;—cj, we must change the w;—values in order to 
change the (2; —Cy )-values. Let (p, g) be an out—of-kilter arc in G’, and let XY 


be the set of nodes in G’ that can be reached from node q along some paths in 
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G'. Hence, X is the set of nodes in the tree 7 constructed earlier. Let 
X =. / —X. where. / ’= {l,...,m}. Note that neither XY nor X is empty when 
we pass to the dual phase, since g ¢ X and pe_X. For (p, g) = (2, 3) in Figure 
11.6 we have X = {3, 4, 1} and X = {2}. In Figure 11.7, we illustrate the sets X 
and X. 

We would like to change the w;—values so that no kilter number is 


worsened and the set X gets larger periodically. If at least one node comes into X 
at finite intervals, then eventually p will come into X and a circuit will be 
created in G’. We have implicitly assumed that X will not get smaller. To ensure 
this, we should change the w;—values so that all arcs having both ends in X are 


retained in the modified graph. 


Consider zj —cj =w;,-w,;—cy. If w; and w; are changed by the same 
amount, then zy —cj remains unchanged. Thus, we can ensure that the set XY 


will contain at least all of the same nodes after a dual variable change if we 
change all of the w;—values in XY by the same amount @ Suppose that we leave 


the w;—values in Y unchanged. Then the only arcs that will be affected will be 


ares going from X to X and from X to X. Specifically, if @> 0 and we change 
the w; —values according to 


Ww; ie X 


’ ie +86, ie X 
W; = — 
then the revised (2 —cy )-values are given by 
(2g — Cy)! = 2y — Cy if ieX,jex 
or ie X,jeX. 


However, if ic X and j < X , we get 


ie ——— a 3)\ xX 

Figure 11.7. X and X inG for (Pp, 9) = (2, 3) in Figure 11.6. 

(zy -cy)’ = (wj+@)-w;-c 
(zi _ cy) + 0. 


ij 


Also, for ie X and je X we get 
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(zy ey: = w—(w; +0)—cy 


Thus, arcs from X to X will have their (245 — Cy )-values increased by 6, 
and those from X to X will have their (24 — Cy )-values decreased by 6. We 
must determine @ so that the kilter number of no arc is worsened and the kilter 
state of some arc is changed. First, we must identify the arcs that can be in the 
set (X, X) and in the set (X,X). (The notation (X, Y) represents the set S = {(x, 
y): xe X, yeY}. The set of arcs in G going between X and X, in either 
direction, are said to constitute the cut with respect to X and X, and will be 
denoted by [X, X]. Hence, [X,X]=(X,X)U(X,X).) 

Examining Figure 11.4, we see that the set (¥,¥) cannot contain an arc 
associated with the kilter state x; < £ y and z;—cy <0, since such an arc (i, /) 
in G would become an arc in G' with the result that if i can be reached (along a 
path) from q, then j can be reached from qg, and thus j €.X (a contradiction). 
Examining the remaining kilter states, we find that the only candidates for 
membership in (X,X) are those identified in Figure 11.8. Recall that arcs from 
X to X in G have their (24; — cj )—values increased. Thus, these arcs change 
kilter states in a left-to-right fashion as indicated in Figure 11.8a. Examining an 
arc from X to X in G that has Xy > Uy and zy — oy < 0, we see from Figure 11.3 


that as @ increases, Ky decreases from Ko xi - t| to Ky = Ixy -uy| and 


thereafter remains constant. Thus, for such an arc, we can increase 0 as much as 
we like and the arc’s kilter number will never increase. Hence, such an arc gives 
rise to an upper limit on 6 of 0, as indicated in Figure 11.8a. Any arc from_X to 


X in G that has Xy = Uy and zy Cy < 0 will have its kilter number first 


decrease and then remain unchanged as @ increases (why?). Thus, again oo is an 
upper limit on the permitted change in @ for such an arc to ensure that no kilter 
number will worsen. However, examining an arc from X to X in G that has 0 


< xy < Uy and zy — Cy < 0, we see that the associated kilter number Ky first 
decreases (to zero), then starts to increase. In order to eliminate the potential 


on @. Similarly, we 


increase in Kj for the arc we must place a limit of |z;; — cj 


must place a limit of |z —c; on @ for arcs having x; = £,; and z,, —c <0. This 
analysis justifies the entries in Figure 11.8. Each of the possible cases for arcs in 
(X,X) is graphically portrayed in Figure 11.8b. 

A similar analysis of arcs from X to_X in G gives rise to the information 
in Figure 11.9. 
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Insofar as worsening of kilter numbers is concerned, Figures 11.8 and 11.9 
indicate that we need only compute @ based on arcs from X to X having xij < 


Ui and arcs from X to X with sg Liye However, if we proceed to define a 


method of computing @ based only on these considerations, difficulties would 
arise in interpreting the meaning of the value @ = oo. Matters are greatly 


simplified if, instead of strict inequalities on flow (that is, x;; iy < Uy and x; > 


¢), we admit weak inequalities on flow (that is, x; < uj and x, 2 ¢,). The 
reason for this deviation from intuition will become apparent when we proceed 
to establish convergence of the algorithm. 


Implied 
2 — Cy <0 2 — Cy =O 2 — Gj > 0 


(WN 78 ij ij _ limit ond 


oo ane ul 


| (b) 
Figure 11.8. Possible kilter states for arcs from X to X in G and limits on @ 


» 2y — Cy 
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Implied 


limit on 6 
+ 


bs “i aap 4 
eB aay ~ 


.. J 


Figure 11.9. Possible cases for arcs Wee: X to Xin G and limits on 6. 


Ee 


The previous discussion concerning limits on @ based on kilter number 
considerations and on (yet to be established) convergence properties leads to the 
following formal procedure for computing @. 

In G define S; and Sz by 

Sa{G, frie X, jf eX, zy —C; y <9,%y Suy} 


and 
S_ ={G,/): ie X,jeX,z; — Cy > 0, xy 2 Ly}. 
Let 
§ = minimum m {zy -c;[} 
(,jeS 


Zi i ey 


6 = minimum {4,0}, 
where 6,=0 if S; is empty. Thus, @ is either a positive integer or 0. These 
two possibilities are briefly discussed. 
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Case 1: 0< @< 0, 


In this case, we make the appropriate changes in w, (that is, w; = w, + Oif 


ie X and w; = w, if ie X ) and pass to the primal phase of the algorithm. 


Case 2: = 0, 


In this case, the primal problem has no feasible solution. (We shall show this 
shortly.) 
This completes the specification of the dual phase of the out—of-kilter 
algorithm and provides the foundation of the overall out—-of-kilter algorithm. 
As an illustration, consider the example of Figure 11.1 with the current 
solution specified by Figures 11.6 and 11.7. Here, 
S, = {(1,2)},  @ =|-2| =2 
52 = {2,3}, & = [3] =3 
O= minimum {2, 3} = 2. 
This gives rise to the following change in dual variables: 
we=wt+0=2 
Ww) = w, =0 
Ws =w3+0=2 
Wy = wy tO=2. 


The x;;—values and the new (z;; — cj; )—values are given in Figure 11.10a. Passing 


to the primal phase of the out-of-kilter algorithm, we see that G’ in Figure 
11.10b contains a circuit involving arc (2, 3), and so we may change the flows. 
The remaining iterations are not shown. 


There is really no need to work directly with the dual variables them- 
selves since we may transform the (z,; —c¢;;)—values directly as follows: 
(244 — Cy if ie X,jeXorie X,jeX 
(24 —¢y)' =4(zy Gy) +9 if te X,feX 
(24, —¢y)-8 if Pex, fexX, 
In Exercise 11.18 we ask the reader to show how the dual variables can be 
recovered from these (z;;—c;;)—values anytime we need them. Note that the 


(Z;; — cj; ) —values are integral (why?). 
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Permitted 
flow 


yf change 


(a) G () G 


Figure 11.10. The new solution obtained from Figure 11.6 after the first dual 
variable change. 


As an example of infeasibility, consider the example network of Figure 


11.11a. Selecting a set of x,;— and w;—values, we find in Figure 11.11b that arc 


(2, 1) is out-of-kilter. Setting up G’ in Figure 11.11c, we find no circuit con- 
taining the arc (2, 1). In this case, X¥ = {1} and X = {2}. Here, S; =© (the 
empty set) and S, =©, and thus @= o. It is clear by examining > and £5, 
that no feasible solution exists. 


Infeasibility of the Problem When 0= 0 


Suppose that during some application of the dual phase of the out—of-kilter 
algorithm, we reach the case where 6 = 0. When this occurs, we must have S, 
= S, =©. Since S; =@, then by reviewing the definition of S,, we conclude 


that ic XY and je X imply one of the following cases: 


abs Zip — Ci <0 and Xi > Ui; 
2. 2ij 7 oy =0; 
3. 2 —ey > 0. 


From Figure 11.8 and since ie X and j € X, possibility (2) or (3) can hold only 
if xj 2 uj. Hence, S;=© holds true only if x; > uy for ie X and feXx. 
Similarly, S; =@ holds true only if ic ¥ and je X implies that xy S by. 
Hence, S; = S2 =© implies 

x, 2u,; if ieX,j7eX (11.6) 
and 

x, Sf, fo iex,jeXx. (11.7) 
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(f,u,c) 
G50 5a 
(ry = =4( (0) wy =0 
ee PAST Ww 22-12 =0 2 
(0, 2, 1) X42 = Oo; 
(a) G (6) w,,x,,and z; —c, 
Permitted flow 
ye change 
1 
4 
CO +Goo: 
5 ; 
() G' (d) XandX 


Figure 11.11. An example of an infeasible network. 


In particular, consider the out—of-kilter arc (p, g) in G’. If (p, q) is in G, then by 


Equation (11.7) we have xp < ¢ ,,. Suppose that x,, = ¢,4. Since (p, g) is out— 


of-kilter, then z > 0, violating the assumption that S, =@. Thus, 


pq “pq 
Xpg < é pq: If, on the other hand, (q, p) is in G, then by a similar argument, we 


may show that x, > Ugp- Thus, at least one of the inequalities (11.6) or (11.7) is 


qP 
strict. Summing these two oe we get 
DE ta a a 2 My 2 by: (11.8) 
iceX iceX iceX ieX 
jex JEex ex JEX 


Since the current flow given by the x, —values is conserving, then the equality 


constraints in Problem (11.1) hold true. Noting that the node set consists of X UX 
and that XY ~X =@, these conservation of flow constraints can be written as 


py xy t pen — 2D xy -— x= i=1,...,m. 


JEx jexe SEX jeX 
Summing these equations over ie X, we get 


XL xt DL xy- L xy - YL xz =O. 


ieX ieX jen ieX 
jex ick icX jeX 
Noting that 
L y= LX; 
ieX JEex 
jeX 1eX 
and that 
De: X ji = =4 x Xi» 
ieX ieX 
jeXx SEX 


the foregoing equation reduces to 
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Y xy - L xy =0. (11.9) 
jex icX 
SEX jex 
Substituting in Equation (11.8), we get 
O> DY uy- ¥ by. (11.10) 
ieX ieX 
jeX SEX 


Suppose by contradiction that there is a feasible flow represented by x, for i, j 


= I,...m. Therefore, uj; > Sy and —é;; 2 —Xys and so, Equation (11.10) gives 
OF Duy = Dts 2 ay he ay (11.11) 
ieX ieX 1eX ieX 
jex SEX jeX jex 


But since the %,;—values represent a feasible flow, they must be conserving. In a 


fashion similar to Equation (11.9), it is clear that the right-hand-side of the 
inequality in (11.11) is equal to zero. Therefore, Equation (11.11) implies that 0 
> 0, which is impossible. This contradiction shows that if 9 =00, there could be 
no feasible flow. 

Note that if we had defined S, and S> by strict inequalities on x, (namely, 


Xy < uy and xy > e,. ip respectively), we could not have produced the strict 


inequality eat in Equation (11.8). 


Convergence of the Out—of-Kilter Algorithm 


For the purpose of the following finite convergence argument we make the 
assumption that the vectors 4, u, and ¢ are integer—valued. 

In developing a finite convergence argument for the out-of-kilter 
algorithm, there are several properties of the algorithm that should be noted. 
First, every time a circuit is constructed in G’ containing an out—of-kilter arc, 
the kilter number of that arc and of the total network is reduced by an integer 
(why?). We can construct only a finite number of circuits containing out—of— 
kilter arcs before an optimal solution is obtained (why?). Second, after each dual 
variable change, the kilter state of each arc in G that has both ends in X remains 
unchanged. Hence, if (p, qg) is not in kilter, then after a dual variable change, 
each node in X before the change is in X after the change. Two possibilities 
exist. One possibility is that a new node k may be brought into _X by virtue of an 
arc being added in G’ from some node in_X to node k. Each time this occurs the 
set X grows by at least one node. This can occur at most a finite number of times 
before node p becomes a member of X and a circuit is created containing (p, q). 
Thus, if the algorithm is not finite, it must be the case that an infinite number of 
dual variable changes take place without the set XY increasing or 0 equalling oo. 
We shall show that this cannot occur. 

Suppose that after a dual variable change, no new node becomes a member 
of X; that is, X does not increase. Then, upon passing to the next dual phase, we 
have the same sets Y and Y and the same x, —values. In addition, each arc 
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from X to X has had its 2y — Cy increased and each arc from X to X has had 


ij 
its 2y — Cy decreased. Thus, after the dual variable change, the new sets S; and 
S5 satisfy 
S;cS, and SicS, 

(why?). Furthermore, by the choice of the (finite) value of @, at least one arc has 
been dropped from either S; or S. Thus, at least one of the foregoing inclusions 
is proper. Now, S,; and Sy may decrease at most a finite number of times before 
S, US; =©@ and @ = occurs, in which case the algorithm stops. 


This completes a finiteness argument for the out—of-kilter algorithm. We 
now summarize the algorithm and present an example. 


11.3 SUMMARY OF THE OUT-OF-KILTER ALGORITHM 


The complete algorithm consists of three phases: the initialization phase, the 
primal phase, and the dual phase. 


Initialization Phase 
Begin with a conserving (integer) flow, say, each x, = 0, and an initial set of 


(integral) dual variables, say, each w; = 0. Compute z) —cy = w;—w, — Gj. 


Primal Phase 


Determine the kilter state and the kilter number for each arc. If all arcs are in 
kilter, stop; an optimal solution has been obtained. Otherwise, select or continue 
with a previously selected out-of—kilter arc. From the network G construct a 
new network G' according to Figure 11.4. For each arc (i, /) in G that is in a 
kilter state that permits a flow increase, place an arc (i, /) in G’ with a permitted 
flow increase, as indicated in Figure 11.4. For any arc (i, 7) in G that is in a kilter 
state that permits a flow decrease, place an arc (/, {) in G’ with the permitted 
flow as indicated in Figure 11.4. For those arcs in G that are members of states 
that permit no flow change, place no arc in G’. In G’, attempt to construct a 
circuit containing the selected out—of-kilter arc (p, q). If such a circuit is 
available, we have a breakthrough. Determine a flow change A equal to the 
minimum of the permitted flow changes on arcs of the circuit. Change the flow 
on each arc of the associated cycle in G by the amount A using the orientation 


specified by the circuit as the direction of increase. In particular, let xj, = xj + 


A if (i, 7) was a member of the circuit in G’; let xj, = x; — A if (, 2) was a 


member of the circuit in G’; let Xi = x; otherwise. Repeat the primal phase. If 


no circuit containing arc (p, q) is available in G’, we have a nonbreakthrough, 
and we pass to the dual phase. (Note that the construction of the tree T described 
in Section 11.2 can be used to detect breakthroughs or a nonbreakthrough, in 
lieu of actually constructing the graph G’.) 
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Dual Phase 


Determine the set of nodes X that can be reached from node qg along a path in 
G'. (This is available via the tree T constructed as in Section 11.2.) Let 


X=./ —X.InG, define S,; and Sy by 
Sy H{G sie Xf EX,2y cy <0, x5 Suy} 


Sy ={(i, /):i€ Xf EX,z Zz ~ Cy > 0, xy 2 Lz}. 


Let 


9 = minimum {{2, Oy 
GES, US 


If G=00, stop; no feasible solution exists. Otherwise, change the w,—values 
and the corresponding (245 —cy) —values according to: 
wast? if ieX 


t WwW; if ie X 


(2 —cy) if G,j)e(X,X)U(X,X) 
(zy —¢y)' = 4 (zy cg) +O if (i,j) €(X,X) 
(z;-cj)-9 if GAEX,X) 


and pass to the primal phase. 


11.4 AN EXAMPLE OF THE OUT-OF-KILTER ALGORITHM 


Consider the network given in Figure 11.12. Initializing the out—of-kilter algo- 
rithm with each x,; = 0 and each w, = 0, we get the sequence of primal and dual 


phases given in Figure 11.13. 


Figure 11.12. An example network. 
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(, 9) = (1, 2) 
C= {(1, 2), (2,4), (4, 5), (5, Y)} 
A=2 


(2.9) =G,4) 

X = {1,2,4,5},X =G} 
5, = {2,3}, 52 ={3,4)} 
0=3 


x=2 ey 
G 


(b) Nonbreakthrough and the First Dual Phase 


(3, 4) is in-kilter 
(p,q) =(4,5) 
X={1,5},¥ =(2,3,4} 
Sy ={(,2),5,2)} 


Sp = {(4,5)} 
0=1 
(c) Nonbreakthrough and the Second Dual Phase 
W2 =3 
x 2=2, x33 =0 


X94 = 2, x34 =0 
x45 = 2, x51 =2 


x59 =0,z*=0 


ws =4 wy =3 


(d) An Optimal Solution 
Figure 11.13. The out—of-kilter method solution for Figure 11.12. 
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11.55 A LABELING PROCEDURE FOR THE 
OUT-OF-KILTER ALGORITHM 


Either for hand or computer calculations there are simple and convenient ways 
to maintain the information required to solve a minimal-cost flow problem by 
the out—of-kilter algorithm. Suppose that we associate with each node / a label 
L(j) = (ti, 47). A label (i, Aj) indicates that the flow on arc (i, /) could be 


increased by an amount A ; without worsening the kilter number of any arc. A 
label (—i, A;) indicates that the flow on arc (j, 7) could be decreased by an 
amount A; without worsening the kilter number of any arc. Note that A; repre- 


sents the current estimate of the amount of flow change that can take place along 
some cycle containing an out-of-kilter arc and either arc (7, /) or (7, 7) in such a 
way that the kilter number of no arc is increased. The labeling algorithm 
becomes as follows. 


INITIALIZATION STEP 
Select a conserving flow, for example, each xy = 0, and a set of dual variables, 


such as each w; = 0. 


Main Step 


1. Ifall arcs are in kilter according to Figure 11.2, stop; an optimal solution is 
obtained. Otherwise, select (or continue with a previously selected) out—of- 
kilter arc, say (s, 4). Erase all labels. If (s, 4) is in one of the states where a 

flow increase, A,,, is required according to Figure 11.4, then set g =¢, p=s, 
and L(q) = (+p,A,,). Otherwise, if (s, ¢) is in one of the states where a flow 
decrease, A,,, is required according to Figure 11.4, then set g = s, p = 1, 
and L(q) = (—p. Agr ). 

2. If node i has a label, node / has no label, and flow may be increased by an 
amount Aj; along arc (7, /) according to Figure 11.4, then assign node j the 


label L(7) = (+ i,4;) where A; = minimum {A;,A;;}. If node i has a label, 
node j has no label, and flow may be decreased by an amount A ,; along arc 
(j, 7) according to Figure 11.4, then give node / the label L(/) = (-i,A;) 
where A ; = minimum (A;,A ;;). Repeat Step 2 until either node p is labeled 


or until no more nodes can be labeled. If node p is labeled, go to Step 3 (a 
breakthrough has occurred); otherwise, go to Step 4 (a nonbreakthrough 
has occurred and the labeled nodes are in the tree 7). 
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3. Let A=A,. Change flow along the identified cycle as follows. Begin at 
node p. If the first entry in L(p) is +k, then add A to Xkp- Otherwise, if the 
first entry in L(p) is —k, then subtract A from x,,. Backtrack to node k and 


repeat the process until node p is reached again in the backtracking process. 
Go to Step 1. 


4. Let X be the set of labeled nodes and let ¥ =. / ‘—X. Define S, ={@ J): 


ie X, ii @ Bey <0, Xj Suy} and S> = {(i, /): ie X, TEX, Zi = Cy 


> 0, xy 2y}- Let @ = minimum{|z, -cj|, 0: Gs) eS, USp}. If 0 =, 
stop; no feasible solution exists. Otherwise, let 

,_jwer+O if ieX 

i |W; if ie X 


and 


(24 - cy if (i, j)e(X,X)U(X,X) 
(24 —cy) =4 (zy —Gy) +9 if GADE(X,X) 
(2y-cj)-9 if (,j)e(X,X) 
and return to Step 1. 


An Example of the Labeling Algorithm 


We shall illustrate the labeling method for the out-of-kilter algorithm by 
performing the first two iterations represented in Figure 11.13a and b. From 


Figure 11.13a we find that arc (1, 2) is an out-of-kilter arc whose flow must be 
increased. 


The sequence of operations of the labeling algorithm are as follows: 
(s,) = (1, 2),9=2,p=1,L2)=(¢ 1, 2). 

L(4) = (42, 2). 

L(5) = (+4, 2). 

L(1) = (45, 2). 

Breakthrough: A = 2. 

L()=+5 => x5] = X51 +A=2. 

L, (5) =+4 > x45 = X45 +A=2. 

L,(4) =4+2 > X94 = X74 +A=2. 

9. L,(2)=+1> x12 =X. +A=2. 

10. Erase all labels, (s, 4 =(@, 4), g = 4, p = 3, L(4) = (4 3, 4). 
11. L(S) = (+4, 3). 

12. L(1) = (45, 3). 

13. L(2) = (+4, 2). 

14. Nonbreakthrough: X = {1, 2, 4, 5}, ¥ = {3}, 0=3. 

15. We= Ww. = Wy = Ws = 3,43 =0. 


SN AAAWN- 
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Since arc (3, 4) is now in-kilter, we select another out—of—kilter arc, erase all 
labels, and continue. 


11.6 INSIGHT INTO CHANGES IN PRIMAL AND 
DUAL FUNCTION VALUES 


It is instructive to see how. the primal (penalty) and dual objective function 
values change in the primal and dual phases of the out—of-kilter algorithm. Not 
only does this provide an alternative convergence argument in which we need 
not necessarily work on the same out-of-kilter arc until it comes in kilter, but it 
also provides an insight into other acceptable ways of modifying primal and 
dual solutions. 

First, consider the primal phase. Define for each arc (i, /) in G a function 
P; (xj) that measures the infeasibility of a conserving flow x as follows: 


P(x) = maximum {0, £;; — 94; } + maximum {0, 4 — 4) }. 


Note that Fy (x4; ) = 0 if and only if £5; < x; < uj, and is positive otherwise. 


Construct a primal penalty functionf{x) defined for any conserving flow x as 
follows: 


f(x) =cx+M Y By) (11.12a) 
GA) 


where M is sufficiently large (for example, M>2 > |qi;\max {uy — £5, uj |, 
GJ) 


le i|}, assuming that —ce < £;; < uj <e and that the algorithm is initialized with 


xj = 0 for all (, j). Observe that the function <x) composes the original objec- 


tive function with a penalty term and is similar to the big—M objective function. 
For x restricted to be flow-—conserving, we have Ax = 0 and so, wAx = 0. 
Subtracting wAx = 0 from the expression in Equation (11.12a) and noting that 
(¢ ~ wA) has components (¢; — zj;), we can equivalently rewrite this function as 

SOD= ¥ (cya y+ MX BCG) (11.12b) 

ij) Gj) 

for any conserving flow x. Whenever we have a breakthrough in the primal 
phase, we modify the flow in a cycle in G such that no kilter number worsens, 
and the kilter number of at least one arc strictly reduces. In particular, no F(x) 


term increases (why?). If any such term decreases (by an integer), then since M 
is large enough, so does the value of f{-). On the other hand, if no Aj(2;) term 


decreases in the cycle, then we must have ti S xy S Uy for all arcs (i, f) in this 


cycle (why?). Thus, the penalty term in Equation (11.12) remains constant 
during the flow change. However, from Figure 11.4 and the foregoing fact, if 
any arc (i, j) in the cycle satisfies 2; —c¢j <0, then we must have 5 < xj < uy 


and x; must be decreasing in the breakthrough. Similarly, if 2; —¢; > 0 for any 
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arc (i, f) in the cycle, then we must have ¢;; < x; < uj and x; must be increasing 


in the breakthrough. In either event, the first term in Equation (11.12b) strictly 
falls. Because we cannot have ay — Cy = 0 for all arcs (i, f) in the cycle in this 
case (why?), we again obtain a strict decrease in f(:). Therefore, f-)falls by an 
integer at every primal breakthrough. 

Next, consider the dual phase. Note from Equations (11.4) and (11.5) that 
hy — vj — (4% —e¢y ) = 0 for all (i, f). Using the current primal flow x, we can 
equivalently rewrite the dual objective function as follows: 


X XylCy — zy t L (Cy — pvp t DV (Gy — yp) H- (11.13) 
(i,j) @j) @J) 


During the dual phase, assuming that @< °°, we note that z; —cj increases by @ 
for all (i, j) € (X,X), and decreases by @ for all (i, j)e (X,X). Hence, from 
Equation (11.9), the first term in Equation (11.13) remains constant (why?). 
Next, consider an arc (i, j)e (X,X). Referring to Equations (11.4) and (11.5) 
and Figure 11.8, suppose that Cy = yj ~ Cy <9. Since ¢j increases by @ we have 


that v4; decreases by @ and hy remains zero in case £4 < xy S uy, and vy 
decreases by min {0, lex} and hy increases by max {0, a-|e;)} in case Xj > 
uj. Therefore, the corresponding term (£4; — x4) vy + (2%; — uj) hy in Equation 


(11.13) remains unchanged if and only if My = é,., that is, arc (i, f) is in-kilter, 


ij? 
and increases by a positive integer otherwise. Furthermore, if z —cj > 0, then 


hy increases by 8, Vij remains zero, and we have xy 2 Uy- Consequently, the 
corresponding term in Equation (11.13) remains unchanged if and only if xy = 
uj, that is, arc (i, f) is in-kilter, and increases by a positive integer otherwise. 
Similarly, consider (i, j)é (X,X). If Cj =2y — Cy 7 0, then h; falls by @and vw; 
remains zero in case £;; < x; < uj, and hy falls by min {8, ¢} and vj increases 
by max {0, @-¢;;} in case x; < 4. On the other hand, if 2, —¢y < 0, then vj 


increases by @ and hy remains zero, while we have x; < ¢,. Again, the 


corresponding term in Equation (11.13) remains unchanged if and only if (é, f) is 
in kilter, and increases by a positive integer otherwise. Because the out—of-kilter 
arc (p, q) is either in (X,X) or (X,X) in G, we obtain a strict increase (by an 
integer) in the dual objective value during any iteration of the dual phase. 
Because the primal penalty function f(-) in Equation (11.12) falls by an 
integer in a primal breakthrough, and the dual objective function (11.13) 
increases by an integer in every dual phase iteration, the difference given by the 
primal minus the dual function values (call it the duality gap) falls by an integer 
every iteration. Hence, if $;=@ and S, = is not realized, the duality gap 
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must become nonpositive finitely. In this event, if Fy) = 0 for all (i, /), that 


is, we have feasibility, then the current solution (x, w) is primal—dual optimal. 
On the other hand, if some P(x) is positive, then since it is an integer, we 
have from Equation (11.12a) that the dual value is > ex + M > 
X«,pley|max { li 
Problem (11.1). Hence, the dual is unbounded and the primal is infeasible, and 
sO we may terminate. 

For the illustrative example of Section 11.4, we compute M> 2(4+6+5 
+ 15 + 12) = 84, say, M= 85. For the starting solution (x,w) = (0, 0), we have 
JO) = M(2 + 2) = 4M = 340 and the dual objective value is 2 — 5 — 12 =-15 
from Equations (11.12) and (11.13), respectively. After the first breakthrough, 
the primal penalty function value becomes 2(1) + 2(—1) + M(0) = 0. Note that 
feasibility is achieved and will be maintained by the algorithm henceforth 
(why?). After the first dual phase, the dual objective value becomes 1(2 — 5) = 
~3, which is an increase of 12 units from —15. After the second (consecutive) 
dual phase, the dual objective value also becomes zero. Thus, we achieve optimality. 

Observe that by keeping track of the duality gap, we need not work with 
the same out—of-kilter arc until it comes in—kilter in order to guarantee finite 
convergence. Furthermore, any algorithmic scheme that ensures that the duality 
gap decreases from its previous lowest value (not necessarily monotonically) by 
an integer at finite intervals, is guaranteed to converge finitely. 


P }. This exceeds any possible feasible value realizable in 


11.7 RELAXATION ALGORITHMS 


We close our discussion in this chapter by providing some elements of another 
highly competitive class of primal—dual methods, known as relaxation algo- 
rithms, for solving minimal—cost network flow programming problems. (Actu- 
ally, this technique can be extended to solve general linear programming problems 
as well.) The approach adopted by this algorithm is to maintain dual feasibility, — 
along with complementary slackness with respect to a pseudoflow, where the 
latter satisfies the flow—bounding constraints but not necessarily the flow 
conservation equations. The method then strives to attain primal feasibility via 
suitable flow augmentations, and in the event of a specific type of primal 
nonbreakthrough following certain flow adjustments, it modifies the dual 
variables in order to obtain a dual ascent. 
To elucidate somewhat further, consider a minimal—cost network flow 
problem that is cast in the following form: 
Minimize {ex : Ax = b,0< x <u} (11.14) 


m 
where A is an m x n node-arc incidence matrix of a connected digraph, >) 5; = 0, 
isl 
and where 0< Uj <2, Vi, J). Examining (7.16) and (7.17), we can write the 
Lagrangian dual to Problem (11.14) as follows: 
Maximize {@(w) : w unrestricted} (11.15a) 
where 
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O(w) = wb + minimum {(¢- wA)x:0<x <u} (11.15b) 
= wb —- maximum {(wA —¢c)x:0<x <u}. 
Note that if we define Cy = 2 — Cy =W; —W; — Cy, VG), as before, we can sim- 
plify the computation of @(w) in Equation (11.15b) as: 


O(w) = wb — > dujmax {0,¢;}. (11.15c) 
(i,j) 


Now, suppose that we have a current dual solution w. (To begin with, we 
can use w =0, or estimate some advanced-start solution.) Let x evaluate 0(W) 
via Equation (11.15b). If this pseudoflow X is a “flow,” i.e., it also satisfies 
Ax =b, then x is a primal feasible solution for which 0(w) = wb +(c— WA)x = 


ex + w(b— AX) = cx. Hence, the dual and primal objective values match, and so, 
X and W are respectively optimal to the primal and dual problems. Otherwise, 
we compute the net excess function e(i) =b; -A;x at each node i = 1,..., m, 


m m 
where A, is the ith row of A. Note that since > 5; = 0 and ¥ A; =0 (why?), we 


i=] i=l 
m 
have that > e(i)=0. However, since Ax #b at this point, some nodes have a 
i=l 
net excess resident supply (e(Z)>0), and some other nodes have a net unsatis- 
fied demand requirement (e(i) <0). Observe also that in the solution of the 
subproblem (11.15b), if Cy > 0, then we must have xy =Uy, and if Cy < 0, then 


we must have x; = 0. However, if c;; = 0, then we have the flexibility of select- 
ing any value for x; ¢[0,u,;]. The relaxation algorithm (so-called because of 


this Lagrangian relaxation framework employed) consequently attempts to begin 
with some node ¢ having e(t) >0, and examines if this excess net supply can be 


dissipated from node ¢ along an incident arc (f, 7) that has Cy = 0, or in the 
reverse direction along an incident arc (/, #) that has Cj, = 0. If saturating these 


arcs (i.e., making the maximum permissible flow augmentations along these 
arcs) yet leaves an excess supply, then a single—-node nonbreakthrough results. 
In this event, the dual variable value w, of this node can be increased to obtain 


a dual ascent, thereby indicating that W was indeed a non—optimal dual solution 
(see Exercise 11.35; in fact, it is the frequent occurrence and the efficiency of 
such single-node dual ascent steps that are largely responsible for the 
computational effectiveness of this method.) Naturally, if this dual ascent leads 
to an indication of unboundedness of the dual problem, then we can declare the 
primal problem to be infeasible and terminate the algorithm. Otherwise, in case 
all the excess e(/) can possibly be dissipated, the procedure dissipates this excess 
only in a fashion such that the receiving node / has e(/) < 0. In this manner, the 
resultant primal breakthrough improves the overall feasibility of the current 
primal solution x (while maintaining it optimal for Problem (11.15b)), without 
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worsening any excess function values (i.e., without driving any of these values 
further away from zero). If e(¢) gets reduced to zero, then we select another node 
that has a positive excess (if it exists; else we have achieved optimality), and we 
repeat this process. On the other hand, if e(f) is still positive, then we begin 
growing a tree JT comprised of nodes having nonnegative excess values and 
connecting arcs that have zero ¢-—values and that have positive residual 
capacities (i.e., would permit a flow change leading away from node #). In this 
process, treating the nodes in 7 as a supernode of the type f in the foregoing 
single-node case, and treating the cut [7,7] as the arcs incident at this 


supernode, we again seek a similar type of dual ascent, or a primal 
breakthrough, or else, grow the tree further. Therefore, while the dual solution is 
not yet optimal, because the excess function imbalances do not worsen and the 


m 
total imbalance Lle| strictly falls finitely often while the dual solution 
i=l 


remains unchanged, we finitely obtain a dual ascent by at least a unit amount. 
Hence, either a dual ascent leads to dual unboundedness (thereby establishing 
primal infeasibility), or we attain dual optimality finitely. Thereafter, we must 
obtain a sequence of primal breakthroughs, resulting in a finite convergence to a 
pair of primal and dual optimal solutions. We refer the reader to the Notes and 
References section for further details on this algorithm. Also, see Exercise 11.43 
for some related dual ascent concepts. 


EXERCISES 


[11.1] Show by manipulating the constraint equations mathematically that any 
minimal—cost network flow problem of the type discussed in Chapter 9 can be 


transformed into the out—of-kilter form (11.1) with, in particular, 0 < ij < Uy < 


oo, V(i, 7), by adding an additional node and at most m additional arcs. Explain 


how you would compute, and use in this context, upper bounds on the values 
that the variables can attain at extreme point solutions. 


[11.2] Consider the minimal-—cost network flow problem: Minimize ex subject 
to Ax = b, £ < x < u, where A is a node-arc incidence matrix. Define a 
conserving flow to be any x satisfying Ax = b (the conservation equations). 
Show that without transforming the network to the out-of-kilter form, the out— 
of-kilter algorithm can be applied directly on the original network with a 
starting conserving flow to solve the problem. 


[11.3] Referring to Exercise 11.2, suppose further that we initialize the algo- 
rithm with a (possibly artificial) conserving flow that is also feasible to the 
bounds. Develop in detail the primal and dual phases of the algorithm. State the 
possible cases that may arise in this specialization. Also, show directly that the 
primal and dual objective values strictly improve during a (primal) breakthrough 
and during a dual phase, respectively. 


[11.4] Solve the following problem by the out—of-kilter algorithm: 
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[11.5] Solve the problem of Exercise 11.4 after replacing (¢3),u31,¢3,) for arc 
(3, 1) by (-l, 5, 5). 
[11.6] Consider the following network flow problem: 


a. Give the kilter state of each arc. 
b. Solve the problem by the out—of-kilter algorithm. 


[11.7] Solve the following problem by the out—of-kilter algorithm: 


[11.8] Show how the out—of-kilter algorithm detects infeasibility in the follow- 
ing problem: 
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(0, 5, -1) 


a. Solve the problem by the network simplex method of Chapter 9. 
b. Transform the problem into a circulation form and solve it by the 
out—of-kilter algorithm. 


[11.11] Consider the following network flow problem having flows as indicated: 


a. Ignoring the fact that the b—values are not zero, apply the out—of— 
kilter algorithm directly to the foregoing network with the starting 
X,~values as given. 


. Solve by the network simplex method of Chapter 9. 
c. Are the solutions of Parts (a) and (b) the same? Discuss! 
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[11.12] Show that after each dual phase we can replace each new w; by w, - 
w;,, where k is some arbitrary node, and the out—of—kilter algorithm remains 
unaffected. (In a computer implementation, we might do this to force one dual 
variable, such as w,, to remain zero and keep al! of the dual variables from 
getting too large.) 

[11.13] How can alternative optima! solutions be detected in the out—of-kilter 
algorithm? 

[11.14] Explain in detail the data and list structures you would use in order to 


efficiently implement the out—of-kilter algorithm. Compare your storage require- 
ment with that for a primal (network) simplex implementation. 


[11.15] In the primal phase of the out—of-kilter algorithm, suppose that we need 
to find a (directed) path from gq to p (in G'), given an out-of-kilter arc (p, q). 


Consider the construction of the usual flow change tree T described in Sections 
11.2 and 11.5. Recall that starting with T = {q}, each node that is included in T 
receives a label equal to the (positive) flow it can receive from g. Suppose that 
at each step we add to T that node, which among all candidates that can be 
connected to nodes already in 7, can receive the largest flow change label. Show 
that if p <7, then this procedure will have found the maximum flow change 


path from g to p in G’. 
[11.16] Is there any difficulty with the out-of-kilter algorithm when ¢,; = uj 


for some (i, /)? Carefully work through the development of the out—of-kilter 
algorithm for this case! 


[11.17] Suppose that we have a feasible solution to Problem (11.1). Assuming 
that the selected arc remains out—of-kilter, is it possible for no new node to 
come into X after a dual variable change? Discuss! 


[11.18] Suppose that we work only with the (z;; —c¢,;)—values after the initial 


dual solution and never bother to change the w;—values. Show how the w,— 
values can be recovered anytime we want them. (Hint: The w;—values are not 
unique. Set any one w; = 0.) 

[11.19] Interpret the dual of the out—of-kilter formulation of Problem (11.1) 
from a marginal—cost viewpoint. 

[11.20] Show that the dual solution given in Section 11.1 is optimal for a fixed 
set of w, —values. 

[11.21] Demonstrate directly that the dual objective function value is unbounded 
as @— o when S; =@ and S, =© during the dual phase of the out—of-kilter 
algorithm. 


[11.22] Considering the out-of-kilter problem, show that a feasible solution 
exists if and only if for every choice of X and X =. / "—X wehave 
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ye 7 2 LX uy 
ieX ieX 
jeX jEex 
(Hint: Review the section on the case where 0= ~.) 
[11.23] Is there any problem with degeneracy in the out—of-kilter algorithm? 
[11.24] If during the primal phase we permit some kilter numbers to increase as 
long as the sum of all the kilter numbers decreases, will the out—of-kilter algo- 


rithm work? How could this be made operational? 
{11.25] Extend the out-of-kilter algorithm to handle rational values of Cis L ip 


and Uy directly. 
[11.26] In the out-of-kilter algorithm, show that if no cycle exists in the subset 
of arcs in G with x #£, and x; # uj, then the current solution corresponds 


to a basic solution of the associated linear program. Indicate how the out—of- 
kilter algorithm can be initiated with a basic solution if one is not readily 
available. Illustrate using the network given below with the indicated conserving 
flow. (Hint: Start with a conserving flow. If a cycle exists among arcs where xy 


xe ij and xy x Uy, consider modifying the flow around the cycle in a direction 


that improves the penalty function value in Equation (11.12).) 


[11.27] Using the results of Exercise 11.26, if the out—of-kilter algorithm is 
initialized with a basic solution to the linear program, show how a basic solution 


can be maintained thereafter. (Hint: Let EF = {(i, j): Xi xe if and xij # Ujy}- 


Start with only appropriate arcs associated with E as members of G’. Whenever 
a circuit exists, change flows. Eliminate any residual cycles in £, as in Exercise 
11.26. Otherwise, after developing X, add an appropriate arc to G’ that is not a 
member of E and that enlarges X; then work with E as much as possible again. If 
no circuit still exists in G’, add another arc that is not in £ but does not enlarge 
X. Continue as often as necessary. If no such arc that does not belong to £ exists 
that enlarges X, then pass to the dual phase. This is an example of block 
pivoting.) 

[11.28] Suppose that we are given a network having m nodes and 7 arcs, all 
lower bounds equal to zero, positive upper bounds, and no costs involved. Show 
how the out-of-kilter algorithm can be used to find the maximum amount of 
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flow from node | to node m. (Hint: Consider adding an arc from node m to node 


1 with £,,, =0, u ©, Cy) =—I| with all other c;;—values set at zero.) 


ml — i 


[11.29] Find the maximum flow in the following network from node | to node 
4 using the out—of-kilter algorithm. (Hint: Refer to Exercise 11.28.) 


[11.30] Suppose that we are given a network having m nodes and n arcs with a 


cost cj for each arc. Assume that there are no negative total cost (directed) 


circuits. Show how the out—of-kilter algorithm can be used to find the shortest 
(least) cost path from node | to node m via the following construction. Add an 
arc from node m to node | having £,,; =U», = 1 and c,,, = 0. Set the lower and 


upper bounds of all other arcs at 0 and 1, respectively. Can this scheme be used to 
find the shortest simple path from node | to node m in the presence of negative 
cost circuits? Explain. 


[11.31] Let cj be the length associated with arc (i, /) in a given network having 


no (directed) circuits. It is desired to find a path with the shortest distance and 
that with the maximum distance between any two given nodes. Formulate the 
two problems so that the out—of—kilter algorithm can be used. Make all possible 
simplifications in the application of the out-of-kilter algorithm for these two 
problems. What is the significance of assuming that the network has no circuits? 
(Hint: See Exercise 11.30.) 


{11.32] An assembly consists of three parts A, B, and C. These parts go through 
the following operations in order: forging, drilling, grinding, painting, and assem- 
bling. The duration of these operations in days is summarized below: 


Duration of Operation 
Part Forging Drilling Grinding __ Painting 


A 1.2 0.8 1.0 0.7 
B 2.3 0.5 0.6 0.5 
C 3.2 1.0 — 0.6 


Upon painting, parts A and B are assembled in two days and then A, B, and C 
are assembled in one day. It is desired to find the least time required for the 
assembly (this problem is called the critical path problem). 


a. Formulate the problem as a network problem. 
b. Solve the problem by any method you wish. 
c. Solve the problem by the out—of-kilter algorithm. 
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d. Solve the problem by the simplified procedure you obtained in 
Exercise 11.31. 

e. Because of the shortage of forging machines, suppose that at most 
two parts can go through forging at any particular time. What is the 
effect of this restriction on the total processing duration? 


[11.33] Provide an interpretation of the primal network simplex algorithm for 
bounded variables in terms of an appropriately restricted execution of the primal 
and dual phases of the out—of-kilter algorithm. In particular, specify the flow 
change A > 0 along with the appropriate circuit in the “primal phase,” the sets X, 


X, and the value @ for the “dual phase.” 


{11.34] For the problem in Equation (11.1), develop an expression for the rate 
in change in dual objective value as the value of the dual variables of nodes in 
some subset X of . / ‘are (marginally) increased. If this rate is positive, how 
would you determine the best possible value by which to increase the dual 
variables of nodes in X? How could you use this information algorithmically? 
(Also, see Exercise 11.43.) 

{11.35] Consider the event of the single-node nonbreakthrough described for 
the relaxation algorithm of Section 11.7. Derive an expression for the amount by 
which the corresponding dual variable value w, can be increased, and show that 
this results in an increase in the dual objective function value, where the dual is 
specified in Equation (11.15). 


[11.36] Suppose that the air freight charge per ton between locations is given by 
the following table (except where no direct air freight service is available): 


A certain corporation must ship a certain perishable commodity from locations 
1, 2, and 3 to locations 4, 5, 6, and 7. A total of 40, 70, and 50 tons of this com- 
modity are to be sent from locations 1, 2, and 3, respectively. A total of 25, 50, 
40, and 45 tons are to be sent to locations 4, 5, 6, and 7, respectively. Shipments 
can be sent through intermediate locations at a cost equal to the sum of the costs 
for each of the legs of the journey. The problem is to determine the shipping 
plan that minimizes the total freight cost. Formulate the problem and solve it by 
the out—of-kilter algorithm. 


[11.37] Coal is being hauled out of Kentucky bound for locations in the 
Southeast, Southwest, Midwest, Northwest, and Northeast. The network of 
routes is given below. 
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Northeast 


Northwest 


Cost 
($/1000 tons) 


Southwest Southeast 


The demands are given by the following chart: 


LOCATION DEMAND (1000s OF TONS) 
Southeast 5 
Southwest 3 
Northwest 10 
Midwest 8 
Northeast 20 


Kentucky has a supply of 65,000 tons per week. In addition to the nonnegativity 
restrictions, there is an upper limit on the flow of 17,000 tons on each arc. Ignoring 
the return route for coal cars, use the out—of-kilter algorithm to find the least 
cost distribution system for coal. 


[11.38] Water is to be transported through a network of pipelines from the big 
dam to the low valley for irrigation. A network is shown where arcs represent 
pipelines and the number on each arc represents the maximum permitted rate of 
water flow in kilo—tons per hour. It is desired to determine the maximum rate of 
flow from the big dam to the low valley. 


Low valley 


Big dam 


a. Formulate the problem so that it can be solved by the out—of-kilter 
algorithm. 

Solve the problem by the out—of-kilter algorithm. 

c. Through the use of a more powerful pumping system the maximum 
rate of flow on any arc can be increased by a maximum of 15 kilo— 
tons of water per hour. If the rate is to be increased on only one 
pipeline, which one would you recommend and why.) 
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[11.39] The “Plenty of Water Company” wishes to deliver water for irrigation 
to three oases: the sin oasis, the devil’s oasis, and the pleasure oasis. The 
company has two stations A and B in the vicinity of these oases. Because of 
other commitments, at most 700 kilo—tons and 300 kilo—tons can be delivered 
by the two stations to the oases. Station A is connected with the sin oasis by a 13 
kilometer pipeline system and with the devil’s oasis by a 17 kilometer pipeline 
system. Similarly, Station B is connected with the pleasure oasis by a 21 
kilometer pipeline system and with the devil’s oasis by a 7 kilometer pipeline 
system. Furthermore, the pleasure oasis and the devil’s oasis are connected by a 
road allowing the transportation of water by trucks. Suppose that the sin oasis, 
the devil’s oasis, and the pleasure oasis require 250, 380, and 175 kilo—tons of 
water. Furthermore, suppose that the transportation cost from station A is $0.06 
per kilo—ton per kilometer, and the transportation cost from station B is $0,075 
per kilo—ton per kilometer. Finally, suppose that the transportation cost between 
the pleasure oasis and the devil’s oasis is $0.25 per kilo—ton. 


a. Formulate the problem so that the out—-of—kilter algorithm can be 
used. 
. Solve the problem by the out—of—kilter algorithm. 
c. Suppose that a road is built joining the sin oasis and the devil's oasis 
with a shipping cost of $0.15 per kilo-ton. Would this affect your 
previous optimal solution? If so, find a new optimal solution. 


[11.40] A manufacturer must produce a certain product in sufficient quantity to 
meet contracted sales in the next four months. The production facilities available 
for this product are limited, but by different amounts in the respective months, 
The unit cost of production also varies according to the facilities and personnel 
available. The product can be produced in one month and then held for sale in a 
later month, but at an estimated storage cost of $2 per unit per month. No 
storage cost is incurred for goods sold in the same month in which they are 
produced. There is currently no inventory of this product, and none is desired at 
the end of the four months. Pertinent data are given below. 


Contracted Maximum —_ Unit Cost of 


Month Sales Production Production 
1 20 40 15 
2 30 50 17 
3 50 30 16 
4 40 50 19 


Formulate the production problem as a network problem and solve it by the out— 
of-kilter algorithm. 

[11.41] Show how a transportation problem and an assignment problem can be 
solved by the out—of—kilter algorithm. 

[11.42] Consider a general linear program of the form: Minimize ex subject to 
Ax =b, £<x <u. Suppose that we begin with a solution x that satisfies Ax = b. 


Develop primal and dual phases of a linear programming algorithm, based on 
the out—of-kilter algorithm, for solving this general linear program. 
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[11.43] Consider the network flow problem stated in Equation (11.14) and its 
dual problem given by Equations (11.15a) and (11.15c), and assume that an 
optimum exists. Let x be a primal feasible solution and let W be an integral m— 
vector. 


(a) Show that O(w) in (11.15c) can be written as 
A= exs DA (w; -w; -cy)- oe pa {0,w; —w; —cj}. (11.16) 
J) 


(Hint: Rewrite wb as wb = (WA —c)X + xX.) 


(b) Consider an out—of-kilter arc (p, g) in G’ and suppose that the primal phase 


results in a nonbreakthrough, yielding the cut [X, X] withg e Xandp < X, 
where X is the set of nodes in the tree T described in Section 11.2. Consider 
augmenting the dual variables w according to: 


w, =, +0, Vie X, and w, =,, Vie X, where 920. (11.17) 
Let h(@) be the function of 0 obtained by substituting (11.17) into (11.16). 
Compute the right-hand derivative of h(@) with respect to 6 at the value 
6=0 (i.e., the rate of change of h with respect to an increase in 0 from the 


value 9=0), and show that this is given as follows, where if = Wi 
w;—cy, VG, J) in this equation: 
athe - = 
|) 2h ES aS a Gee) + 
08 |pg | GDLX) ”  GNeX.X) 
~ 1c <0 cj 20 
(11.18) 
op ae uy—Xy)- XD xij 
CDE x) CDE x) 
Cj >0 


i y= 

(c) Show that the second and fourth terms in (11.18) are zero. What is the 

interpretation of the first and third terms in (11.18) in terms of the kilter 

numbers of the arcs in the cut LX, X ]? Hence show that the expression in 

(11.18) has a value of at least one, thereby yielding a dual ascent of at least 

one unit as @ is increased. By how much can @ be increased while 
maintaining the same rate of ascent as in (11.18)? 

(d) Discuss if it is possible to get a further ascent in the dual objective value 

h(@) if Gis increased beyond the value determined in Part c. Accordingly, 


explain how you might solve the /ine search problem to maximize { h(0) : 0 


2 0} using the expression (11.18). Show that while this might worsen some 
kilter numbers, the out—-of-kilter algorithm can still achieve finite 
convergence when the dual phase is implemented with such a linear search 
strategy. (Hint: Examine the effect on the duality gap.) 
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(e) If the cut [X, X] is not necessarily determined as a consequence of a 


nonbreakthrough in the primal phase, but is arbitrarily selected with both XY 
and X being nonempty, verify that (11.18) still holds true. Hence, discuss 
how you might perform coordinate ascent steps or single-node ascent steps 
(see Section 11.17) in the present context when either X or X is a singleton. 
How would you incorporate such ascent steps within the overall framework 
of the out-of-kilter algorithm, and what is the potential advantage of 
employing such a strategy? 


NOTES AND REFERENCES 


1. 


Fulkerson [1961la] developed the out—of-kilter algorithm for network 
flow problems. For a slightly different development of the out—of-kilter 
algorithm, see Ford and Fulkerson [1962], and for a specialization, see 
Kennington and Helgason [1980]. 

The presentation of the out—of-kilter algorithm in this chapter follows 
that of Clasen [1968], especially the division of states according to values 


of flows x; and reduced costs 2; —cj. 


The spirit of the out-of—kilter algorithm can be extended to a procedure 
for general linear programs. This has been done by Jewell [1967]. The 
corresponding steps in the general case require the solution to linear 
programming subproblems instead of finding cycles or changing dual 
variables in a simple way. 

Barr et al. [1974] provide a streamlined implementation scheme for the 
out-of-kilter algorithm along with computational results. Another implemen- 
tation scheme is described in Singh [1986]. Computational experience 
and comparisons are also provided by Glover and Klingman [1978] and 
Hatch [1975]. 

Bertsekas and Tseng [1988b] describe a primal—dual network flow 
relaxation algorithm that operates two-three times faster on the NETGEN 
benchmark test problems compared with RNET, an effective primal sim- 
plex code developed at Rutger’s University by Professor Grigoriadis and 
Professor Hsu. This algorithm is discussed briefly in Section 11.7. Its 
version RELAX-IV, and extension to solve generalized networks are 
respectively described in Bertsekas and Tseng [1988b, 1994]. The 
principal computational advantage comes from the quick dual ascent 
steps using cuts based on single nodes as opposed to determining steepest 
ascent cuts (see Exercises 11.34 and 11.43). All modern-day commercial 
software have specialized routines to solve network structured problems 
that are being continually refined (e.g., the popular software CPLEX has 
an efficient network simplex code, NETOPT). For extensions to handle 
convex—arc costs, see Bertsekas et al. [1987], and for the design of relaxa- 
tion methods for general linear programs (see Tseng and Bertsekas 
[1987]). Also, for specializations of interior point methods to solve 
network flow problems and computational comparisons with simplex and 
relaxation methods, see Mehrotra and Wang [1996]. 
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TWELVE: MAXIMAL FLOW, 
SHORTEST PATH, 
MULTICOMMODITY FLOW, 
AND NETWORK SYNTHESIS 
PROBLEMS 


Two special and important network flow problems are the maximal flow prob- 
lem and the shortest path problem. Both of these problems can be solved by 
either the network simplex method of Chapter 9 or the out—of-kilter algorithm 
of Chapter 11. However, their frequent occurrence in practice and the 
specialized, more efficient procedures that can be developed for handling these 
two problems provide a strong case for considering them separately. 

We also include in this chapter an introduction to the class of network 
flow problems called multicommodity network flows. In Chapters 9, 10, and 11 
we have considered network flow problems in which it was not necessary to 
distinguish among the units flowing in the networks. There was essentially a 
single commodity or type of unit. There are network flow problems in which 
different types of units must be treated. In these instances, supplies and demands 
are by commodity type, and the distinction among the commodities must be 
maintained. We shall examine this multicommodity flow problem, consider the 
difficulty in dealing with it, and present a decomposition—based procedure for 
solving it. 

Another topic that we introduce is network synthesis. The network flow 
problems considered thus far are analysis problems. A network is given, and 
some analysis is performed on it. On the other hand, the network synthesis 
problem requires one to construct an optimal network (in some defined sense) 
that satisfies certain specifications. Several of the tools developed in this book, 
including network analysis and decomposition methods, can be used to study 
such problems. 


12.1 THE MAXIMAL FLOW PROBLEM 


We begin by presenting Ford and Fulkerson’s /abeling method for solving 
maximal flow problems. This method is simply a specialization of the out—of— 
kilter algorithm applied to the maximal flow problem. Specialized primal sim- 
plex implementations have been found to be significantly faster than this 
method (with a speed-up factor of 1.5—5) and require only one-third of the 
amount of storage. However, this method does provide useful insights, and as 
with the general out-of-kilter algorithm, improved primal—dual implementa- 
tions can make it more competitive. In fact, at the end of this section, we briefly 
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discuss such a variant known as the preflow—push strategy, which is among the 
most efficient (theoretically as well as in practice) methods to solve the maximal 
flow problem. 

To formally describe the maximal flow problem, consider a network 
having m nodes and » arcs through which a single commodity will flow. We 
associate with each arc (i, j) a lower bound on flow of ¢;, = 0 and an upper 


bound on flow of u;. We shall assume throughout the development that the uj— 


values (arc capacities) are finite integers. There are no costs involved in the 
maximal flow problem. In such a network, we wish to find the maximum 
amount of flow from node 1 to node m. 

Let frepresent the amount of flow in the network from node 1 to node m. 
Then the maximal flow problem may be mathematically formulated as follows: 


Maximize f 


in 7 f ifi=1 
subjectto Yxy- Lxy=) 0 ifi*lorm 
j=l k=l -f ifi=m 


xi Suy, i,j =1,...,m 
xij =0, i,j =1,...,m, 

where the sums and inequalities are taken over existing arcs in the network. This 
is called the node-arc formulation for the maximal flow problem since the 
constraint matrix is a node—arc incidence matrix. (See Exercise 12.19 for 
another arc--path formulation.) Noting that f is a variable and denoting the 
node-arc incidence matrix by A, we can write the maximal flow problem in 
matrix form as: 


Maximize f 
subject to (e,, —e;)f + Ax=0 
x<u 


x20. 


Since the activity vector for fis (e,, —e,), the difference of two unit vectors, we 
may view f as a flow variable on an arc from node m to node 1. This provides 
the direct formulation of the maximal flow problem in (out—of-kilter) 
circulatory form (with zero right-hand-side values for the flow conservation 
equations). Recalling that the out—of—kilter problem dealt with minimization, we 
assign a cost coefficient of zero to every flow variable except x,,; =f, which 
receives a cost coefficient of —1. 

Arc (m, 1) is sometimes called the return arc. Figure 12.1 presents an 
example of the maximal flow problem and its equivalent out—of—kilter network 
flow problem. In Figure 12.1 the lower bound £,,,,; = £4; = 0 is derived from the 


fact that all x; = 0 and x,,; = 0 is a feasible solution to the maximal flow 


problem. Thus, the maximal value of x,,, will never be less than zero. 
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Figure 12.1. An example of a maximal flow problem: (a) Maximal flow 
problem. (b) Out—of—kilter equivalent problem. 


Before continuing with the development of an algorithm to solve this 
maximal flow problem, we introduce the useful and important concept of cuts 
(this was briefly alluded to in Chapter 11). 


Cut (Separating Node m from Node 1) 

Let X be any set of nodes in the network such that X contains node | but not 
node m. Let X =. / —X. Then the set of arcs that have one endpoint (head or 
tail) in X and the other in X is called a cut separating node m from node 1, and 
is denoted by [X,X]. (For the sake of interest, we mention here that a cut-set 


for a connected network is a minimal cut in the sense that it is a cut that discon- 
nects the network, but no proper subset of it has this same property.) The set 


(X,X)= {(i, j):i¢ X,j €X} is the set of forward arcs of the cut (separating 
node m from node 1), and the set (¥,X)= {(i, ;):i¢ X, j © X} is the set of reverse 


arcs of the cut (separating node m from node 1). For brevity, we shall refer to 
these as the forward cut and the reverse cut, respectively. 


Capacity of a Cut 


Let [X,X] be any cut in a network G, and let (X,X) and (X,X) be the 
corresponding forward cut and reverse cut, respectively. Then 


uX,XJ= Yd uz fi 


pee oe me ij 

G)e(X,X) Gel X,X) 
is called the capacity of the cut. Note that in our context, we have £ yj =9 forall 
arcs (i, j), and so the associated capacity of the forward cut defined as u(X,X) 
= Li, e(xX,¥)4y equals the capacity of the cut ul X,X]. Hence, we may focus 
on identifying just the forward arcs (X, X) associated with the cut (see Exercise 


12.7 for an extension). In Figure 12.1a there are several forward cuts separating 
node 4 from node | in G. They are: 
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X = {i}, X ={2,3,44 (X,X)={(1,2),(1,3)}, u(X,X)=5 
X = {1,2}, X = 3,4} (X,X) = {(1,3),(2,3),(2.4)}, u(X¥,¥)=9 
X = {1,3}, X = {2,4 (X,X) = {(1,2),(3,4)}, u(X,X)=3 
X ={1,2,33}, X={4 (X,X) = {(2,4),(3,4)}, u(X,X)=5. 


Let [X,X] be any cut separating node m from node | in G. Summing the flow 


conservation equations of the maximal flow problem over nodes in_X, the flow 
variables that have both ends in X cancel and we get 


>; xij L xy =f. (12.1) 

ie X ieX 

jex sex 

Using x; 2 4; = Oand xj < uy, we get 
ae a ee ae ae (12.2) 


< 7 by _ 
(i j)e(X,X) Ge(X,X) Gi )e(X,X) 

This leads to the following. 

Lemma 12.1 


The value f of any (feasible) flow is less than or equal to the capacity u{X,X] of 
any cut (separating node m from node 1). 


The Dual of the Maximal Flow Problem 


Consider the dual of the maximal flow problem: 


. i . - in 
Minimize LL uyhy 
i=l j=l 
subject to Wm —W = 1 
Ww, —w; +hy 20, i,j =1,...,m 


hy 20, i,j =l,...,m, 
where w is associated with the flow conservation equations and h is associated 
with the constraints x < u. Note that the first dual constraint is associated with 


the flow f whose column is e€,,—e,. A typical column a, of the node—arc 


incidence matrix A has +1 in the ith position and —1 in the jth position, which 


leads to the dual constraints w; —w, +hy 2 0. 


Let [X, X] be any cut and consider the foregoing dual problem. If we let 


WwW; = 


0 if ieX 
1 if ieX 


yall if GA)eEX,X) 
y 0 otherwise, 
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then this particular choice of w and h provides a feasible solution to the dual 
problem (why?) whose dual objective is equal to the capacity of the cut. 

Thus, Lemma 12.1 also follows from the duality theorem, which states 
that any feasible solution to a minimization problem has an objective value 
greater than or equal to that of the associated maximization problem. As the 
reader may suspect, we shall show that the capacity of a minimal cut (i.e., one 
having a minimal capacity) is equal to the value of the maximal flow. We shall 
prove this constructively. We ask the reader to prove this result in Exercise 12.8 
using the earlier duality relationship. 


An Algorithm for the Maximal Flow Problem 


From Lemma 12.1, if we are able to find a flow and a cut such that ulX,X] =f, 


we will have the maximal flow (and the minimal cut). We shall do this 
constructively by simply specializing the out—of-kilter algorithm to this 
problem. 

Suppose that we start with any feasible (integer) flow in G, say, each xj 


= 0. From G, we construct G’ as follows: 
1. If arc (i, /) is in G and x; < uj, then we place arc (i, /) in G’ along 
with a permitted flow change value Ay = uj — xj. 
2. If arc (i, /) is in G and xj > 0, then we place are (/, i) in G' with a 


permitted flow change value Aj; = xj. 


Now, in G’, two possibilities exist: 
Case 1 
A path P exists, in G’, from node 1 to node m. 
Case 2 
No path exists, in G’, from node 1 to node m. 


In Case 1, we may construct a new feasible flow having a greater objec- 
tive value. Let A be equal to the minimum permitted flow change on the path P 
from node 1 to node m in G’, that is, A= minimum {A,;: (i, /) is in the path}. 
Note that A is a positive integer (why?). Consider the associated chain P’ 
(undirected path) in G. Construct a new flow as follows. Add A to flows on arcs 
of the associated chain in G that have the same direction as the path in G’, 
subtract A from flows on arcs of the associated chain in G that are against the 
direction of the path in G’, and leave all other arc flows unchanged. The new 
flow is feasible (why?). The value of the new flow is f’ =f+ A (why?). 

Assuming that the capacities are finite, Case 1 can occur only a finite 
number of times before Case 2 occurs (why?). When Case 2 occurs, let X be the 
set of nodes in G’ that can be reached along some path in G’ from node 1. Let 
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X =./ '—X and note that node m belongs to X (why?). Consider the cut 


[X, X], i-e., the arcs in G between X and_X. First, every arc (i, 7) in G from X to 
X must have Xp Uys otherwise, there would be an arc (i, /) in G’ and 7 would 
be a member of X (a contradiction). Second, every arc (i, /) in G from X to X 
must have xj; = 0; otherwise, there would be an arc (j, i) in G’ and i would be a 


member of X (a contradiction). Substituting in Equation (12.1), we get 
Luy-0=f, or u[X,X] = u(X,X) =J 
1eX 


SEX 
Thus, we must have the maximal flow at hand by noting Equation (12.2). 
Hence, we have constructively proved the following. (See Exercise 12.7 for a 
direct generalization when £, #0.) 
Theorem 12.1 (Maximal Flow—Minimal Cut Theorem) 


The value of the maximal flow in G is equal to the capacity of the minimal cut 
in G. 


Summary of the Maximal Flow Algorithm 


The constructive proof of the maximal flow—minimal cut theorem leads to the 
following maximal flow algorithm. 


INITIALIZATION STEP 


Select a set of feasible (integer) flows, say, each x, = 0. 


MAIN STEP 
From G construct G' as follows: 


1. All of the nodes in G are in G’. 
2. If x < uj in G, place (i, /) in G’ with the permitted flow change on 
3. If x; > 0 in G, place (j, i) in G’ with the permitted flow change on 


x 


Note that arc (i, /) in G will give rise to two arcs in G' if 0 < Xj < uj. Attempt to 


locate a path P in G’ from node | to node m. If no such path exists, stop; an 
optimal solution is at hand. Otherwise, let A be the minimum permitted flow 
change on P in G’. Add A to the flows on arcs of the associated chain in G that 
have the same direction as the path in G’, subtract A from the flows on arcs of the 
associated chain in G that are against the direction of the path in G’, and leave all 
other arc flows unchanged. Repeat the main step. 
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Locating a path in G’ from node 1 to node m above is called a 
breakthrough, whereas finding no such path is called a nonbreakthrough. 


An Example of the Maximal Flow Problem 


Consider the network of Figure 12.la. Figure 12.2 presents the complete 
solution to the maximal flow problem for this network. 


Basic Solutions in the Maximal Flow Algorithm 


In Chapter 9, we characterized basic solutions to a network flow problem. 
Recall that a basic solution to a network flow problem consists of a set of 
nonbasic variables at one of their lower or upper bounds plus a set of variables 
that form a rooted spanning tree. Thus, we may conclude that if the set E = {(i, 
Jj): 0 < xj < uy} does not contain a cycle, then we have a basic feasible solution 


at each iteration of the maximal flow algorithm (why?). 

To identify a basis after each flow change in the maximal flow algorithm, 
we take all of the variables in the set E plus an additional number of variables at 
one of their bounds to form a spanning tree. This set, together with the artificial 
variable (located at node m), forms a rooted spanning tree (the nonbasic 
variables are at one of their respective bounds). Note that since f, the flow in the 
network, is a variable, it must be taken onto the left—hand—side of the constraint 
system and becomes an arc from node m to node 1 (as discussed previously). 

In Figure 12.3 we present bases corresponding to the solutions at each 
iteration of the example in Figure 12.2. Notice that the bases in Figures 12.3b 
and 12.3d are unique. Also, notice that in Figure 12.3c no basis is possible that 
corresponds to the maximal flow algorithm solution at that point. In Exercise 
12.9 we suggest a procedure for finding paths in G’ in such a way that we shall 
always have a basic solution available. Finally, notice that the bases presented in 
Figures 12.3a and b are not adjacent. To obtain the basis in Figure 12.3b we 
have replaced two basic variables in the basis of Figure 12.3a by two nonbasic 
variables. This is an example of block pivoting discussed in Chapter 3. 


A Labeling Algorithm for the Maximal Flow Problem 


Either for hand or computer calculations, there are simple and convenient ways 
to maintain the information required to solve a maximal flow problem. As in the 
out—of-kilter method, we shall present a (tree construction) labeling algorithm 
that does not require the creation of the network G’. Suppose that we associate 
with each node / a label L(7) = (+i, A;) containing two pieces of information. 
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(+) P= {(1,2), (2,3), 3,4)} 
A=1 


oF 


, ie 
nD P= {(1,3), (3,2), (2,4)} 
A=1 
4 


ft 


Ul 


> 


Optimal solution: 

f° =3,x2 =¥24=1, 
x33 =0 

X13 = X34 =2 

X = {1,3},X = {2,4} 
(X,X)= {(1,2), (3,43 
u(x, X)=3. 


(d) Nonbreakthrough 


Figure 12.2. The solution for the network of Figure 12.1a. 


The second entry, A ip in L() indicates the amount of flow that can be sent to node 


J from node | through the current network with given flows, without violating 


the capacity constraints 0 < xj< Uj. The first entry, +i, in L(/) indicates the 


previous node in the chain along which flow can be changed. If the first entry in 
LG) is +i, then flow will be added to arc (i, /) ; otherwise, if the first entry is —i, 
then flow will be subtracted from arc (/, i). The labeling algorithm becomes as 
follows: 
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INITIALIZATION STEP 


Set xy = 0 for i, 7 = 1,..., m. 


MAIN STEP 


1. Erase any labels and set Z(1) = (-, «). 

2. Ifnode i has a label, node j has no label, and Xy < uy, then set Z(/) 
= (+i, A; ), where A; = minimum {A;, Uj ~Xy}. If node i has a label, 
node j has no label, and x > 0, then set L(V) = (-i, A jp» where A a= 
minimum{A;, x jit» Repeat Step 2 until either node m is labeled or 


until no more nodes can be labeled. Proceed to Step 3. 

3. Ifnode m is not labeled, stop; an optimal solution is at hand. Other- 
wise, if node m is labeled, then change flows in the network as 
follows. Set A = A,,. Begin at node m and consider the first entry of 
L(m). If the first entry is +k, then add A to x,,,. If the first entry of 
L(m) is -k, then subtract A from x,,;. Backtrack to node k and repeat 
the process until node | is reached. Return to Step 1. 


When the algorithm stops, let X be the set of labeled nodes and 
X =. / —X.The set [X,X] is a minimal (capacity) cut. 
An Example of the Labeling Algorithm 


Suppose that we apply the labeling algorithm to the maximal flow problem of 
Figure 12.1a to produce the first two iterations of the maximal flow algorithm 
represented in Figure 12.2a and Figure 12.2b. We begin with each x = 0. 


The sequence of labeling operations are as follows: 


L(1) =(-, ©). 

L(2) = (41, 1). 
L(3) = (#2, 1). 
L(4) = (43, 1). 


Breakthrough: A = 1. 

L (4) = +3 > x34 =0+A=1. 
1,3) =+2 > x93 =0+A=1 . 
L,(2)=+1> 32 =0+A=1. 
9. Erase all labels, L(1) = (-, «). 
10. £(3) =(4+1, 4). 

li. £(2)=(3, 1). 

12. L(4) =(4 2, 1). 

13. Breakthrough: A= 1. 


BO SP EON Gr OB Od 
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O<x<u 
G The unique basis for G 


G The unique basis for G 


(d) Nonbreakthrough 


Figure 12.3. Comparison between solutions in the maxima! flow algorithm 
and bases (when possible) for Figure 12.2. 
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14. [,(4) =42 > x4 =0+A=1. 
15. L,(2)=-3 > x3 =1-A=0. 
16. L,(3)=+1> 43 =0+A=1. 


Having completed the change of flows, we erase all labels and continue. 
The foregoing sequence of labels is not unique. We selected it because it illus- 
trated the method completely. (See Exercise 12.15 for an implementation 
variant.) 


Some Theoretical and Practical Expedients: Complexity Analysis, a 
Polynomial—Time Scaling Algorithm and a Preflow—Push Implementation 


To begin the present discussion, let us first analyze the complexity of the fore- 
going type of augmenting path algorithm. Letting f” denote the optimal maximal 


flow value and U denote the maximum arc capacity, we know now from the 
maximal flow—minimal cut theorem that 


f <ul. 7 —{3)<(m-DU. 


Because each breakthrough value A is at least 1, the number of primal phases 
performed by the algorithm is therefore no more than (m—1)U. Noting that 


each such primal phase could at most scan all the arcs (n of them, say), the effort 
per iteration is of order O(n), leading to the order of complexity O(mmU) for the 
overall algorithm. Under binary encoding of data, this algorithm is therefore of 
exponential complexity in the size of the problem because U = 2!082U  How- 
ever, it is a pseudopolynomial algorithm, i.e., it is of polynomial complexity in 
the size of the problem, provided we record the size of the data using a stroke or 
unary encoding scheme (i.e., one “stroke” per unit of the integral data values). 
Actually, by using shortest augmenting paths (in terms of the number of 
arcs used to reach node m from node 1), one can show that the augmenting path 
lengths generated are monotone increasing (nondecreasing), and that within 
every n iterations, this path length strictly increases (see the Notes and Refer- 
ences section). Noting that the maximum augmenting path length from node | to 
node m is bounded above by (m — 1), the number of flow augmentations is of 


order O(nm), leading to an overall complexity of O(n? m). This yields a strongly 
polynomial algorithm (i.e.. of polynomial complexity in terms of just the 
number of nodes and arcs in the problem, and independent of the capacity data). 
There is another insightful idea that can be used to convert the out—of— 
kilter maximal flow algorithm discussed in this chapter into a polynomial-time 
algorithm. This is a useful scaling concept that is actually widely applicable to 
more general minimal—cost network flow algorithms (see the Notes and Refer- 
ences section). The main idea here is to explore restricted higher values of 
breakthroughs first before examining progressively smaller values, ultimately 
solving the problem when permitting any breakthrough of value A > 1. More 
specifically, given a particular lower bound A > 1 on the permissible break- 
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through values that will be examined, we perform a corresponding A-scaling 
phase. This step is identical to the primal flow augmenting phase of the out—of- 
kilter algorithm discussed earlier, except that in the graph G’ (or the flow aug- 
menting tree constructed), we include only those arcs whose residual capacity 
(permitted flow change) is at least A. Consequently, if at all a breakthrough is 
obtained, it must necessarily be of value at least A. Furthermore, in the event of a 


nonbreakthrough, if |X, X] denotes the minimal cut separating nodes 1 and m, 
the residual capacity on all the arcs in this cut (where this residual capacity 
equals uj — x; on the forward arcs and x; on the reverse arcs) must be strictly 
less than A (why?). Hence, denoting f,5y as the current value of the maximal 


flow objective function, the optimality gap f* — frow satisfies: 


fr — faow <n. 


With this fundamental concept, suppose now that we initialize the 
algorithm with x = 0 and f,,, = 0, and we begin by considering A = 27, where 
q =| log, (U) |, the rounded—down value of log, U. Observe that this value of 
A is the greatest power of 2 for which we can hope to achieve a breakthrough of 


this level because 27*! > U , the maximum arc capacity in the network. Hence, 


assuming a connected network (so that n > m — 1), and noting the foregoing 
inequalities, we presently have the following upper bound on the current 


optimality gap: 


Cf" — frow) =f” <(m-WU < nU <n29*" = 2nd. 
We now perform the main step as stated below: 


MAIN STEP 


Given A with (f* —fiow)<2nA, perform the A-scaling phase as described 


above. (Note that the current optimality gap implies that the number of break- 
throughs achieved in this phase, each of value at least A, is less than 2n.) If A= 
1, then stop; the current solution is optimal because A = 1 permits all possible 
remnant breakthroughs. Otherwise, we know from before that in the event of a 
nonbreakthrough in the A-scaling phase, the current updated value of the 
maximal flow frow satisfies 


(F" ~Saay)< nd = 2n{ 5 | 


Hence, putting A,ew =A/2, we again have (f* — fiow )<27Anew » and so we repeat 
the main step with A replaced by Ajo. 
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Observe that in this process, we will execute the main step using values 
of A equal to OF a) 3 OP ie. (q + 1) or O(log, U ) times. Because each A- 
scaling phase has fewer than 2 breakthroughs, the effort for each such phase is 


of complexity O(n?) (why?), leading to a polynomial overall algorithmic com- 


plexity of O(n? log, UV). 
We conclude this discussion by mentioning that an even better theoretical 


complexity of order O(mn) (which is essentially of order O(n? ) for dense 


networks) can be achieved by adopting what is known as a preflow—push 
strategy. Besides yielding an improved theoretical complexity, this approach has 
been shown to significantly enhance the effectiveness of solving maximal flow 
problems in practice. Its basic idea is motivated by recognizing that the principal 
problem with standard flow augmenting path algorithms is that it is possible for 
such augmenting flows to repeatedly traverse long chains having relatively high 
residual capacities before hitting some bottleneck arcs at each step, which 
results in relatively small breakthrough values. Hence, the method suggests 
sending excess flows to intermediate nodes as possible, particularly over such 
high capacity long chains, even though not all of this flow will ultimately reach 
node m, where the excess flow to a node (other than the starting and terminus 
nodes 1 and m) is defined as the total inflow minus the total outflow. This is a 
preflow phase. Subsequently, in a push phase, this excess flow is either sent 
forward to node m as possible, or, when any remnant excess accumulations 
cannot be pushed forward, the excess flows are ultimately sent back to node 1. 
In this fashion, the augmenting flows typically tend to traverse the 
aforementioned types of long chains only twice: once in the forward preflow 
step, and once in the reverse direction during the push phase. We refer the 
reader to the Notes and References section for further details on this algorithm, 
as well as for an alternative scheme based on pseudoflows (where nodes can 
have excess or deficit flows), which yields an improved theoretical complexity 
bound of O(@mnlog(m)). 


12.2 THE SHORTEST PATH PROBLEM 


Suppose that we are given a network G having m nodes, n arcs, and a cost cj 


associated with each arc (i, 7) in G. The shortest path problem is: Find the 
shortest (least costly) path from node | to node m in G. The cost of the path is 
the sum of the costs on the arcs in the path. 


A Mathematical Formulation of the Shortest Path Problem 


We may think of the shortest path problem in a network flow context if we set 
up a network in which we wish to send a single unit of flow from node | to node 
m at minimal cost. Thus, 5, = 1, 5,, = —I, and 6; = 0 for i # 1 or m. The 


corresponding mathematical formulation becomes: 
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. . . m m 
Minimize 2 2 Cy Xi 
i=1 j=l 
1 ifi=1 


m m 

subject to 2 Mp y= 0 ifi#lorm 
j=l k=l -l ifi=m 
Xj =0orl i,j=l,..,m, 


where the sums and the 0—1 requirements are taken over existing arcs in G. The 
constraints x,; = 0 or | indicate that each arc is either in the path or not. 


Ignoring the 0-1 constraints, we again find the familiar flow conservation 
equations. From Chapter 9, we know that the node—arc incidence matrix associ- 
ated with the flow conservation equations is totally unimodular. Consequently, 
if we replace the binary restrictions on x, by x, 2 0, then if an optimal solution 


exists, the simplex method will automatically obtain an integer basic feasible 
solution where the value of each variable is either zero or one. Thus, we may 
solve the integer program as the following linear program: 


. . . m m 
Minimize > > Cy Xj 
i=l j=l 
ni in ] if i=] 
subject to > xy DL xyyz=) O ifi#lorm 
j=l k=l -| ifi=m 
xy 2 0, i,j =1,...,m. 


Because the shortest path problem is a minimal—cost network flow prob- 
lem, we can solve it by one of the methods described in Chapters 9 and 11. 
However, we shall soon see that more efficient methods exist for this problem. 
Consider the dual of the shortest path problem: 


Maximize w—W,, 


subject tow, —W; Sey i,j=l,..,m 
w; unrestricted, i=1,...,m. 


It will be more convenient to make the substitution w; = —w;. As we shall 
shortly see, w; — w is the shortest distance from node | to node i at optimality. 


Hence, we can determine the shortest distance from node | to all nodes of the 
network. 


A Shortest Path Procedure When All Costs Are Nonnegative 


Consider the case when all cj > 0. In this case, a very simple and efficient 


procedure, known as Dijkstra’s algorithm, exists for finding a shortest path 
(from node | to node m). This method also automatically yields shortest paths 
from node | to all of the other nodes as well. 
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INITIALIZATION STEP 
Set wj =Oand let X= {1}. 
MAIN STEP 


Let X¥ =. / '—X and consider the arcs in the set (Y,X¥) = {(i,j):ie¢ X, 
je X}. Let 


+c,, = minimum {w; +c;;}. 
ppg oy a i 
G)ACX,X) 


Set Wo = Wy +Cpyg and place node qg in X. Repeat the main step exactly m — | 
times (including the first time) and then stop; an optimal solution is at hand. 


(Because of the fact that whenever the label of a node q is set at wi = Ww, +Cyg 


as above, then this turns out to be the distance of the shortest path from node 1 
to node q as established next, we refer to such a procedure as a labeling—setting 
algorithm.) 


Validation of the Algorithm 


We now prove that the algorithm produces an optimal solution. Assume, 
inductively, that each w; for i e X represents the cost of a shortest path from 


node | to node i. This is certainly true for i = 1 (why?). Consider the algorithm 
at some point when a new node q is about to be added to X. Suppose that 


W,+Cyq = minimum {w; +¢;;}. 12.3 
Cie Sa oe 
We shall show that a shortest path from node | to node q has length 


Wa = Wp +Cpg and can be constructed iteratively as the available shortest path 


from node | to node p plus the arc (p, g). Let P be any path from node | to node 
q. It suffices to show that the length of P is at least w,. Since node 1 is in X and 


node q is currently in X, then P must contain an arc (i, 7) where i ¢ XY andj ¢ ¥ 


(here, i and j could be p and gq, respectively). The length of the path P is thus 
equal to the sum of the following: 


1. The length from node | to node i; 
2. The length of arc (i,j), that is, ¢;,; 


3. The length from to q. 
By the induction hypothesis, the length from node 1 to node i is greater than or 
equal to w;. Because the costs of all arcs are nonnegative by assumption, then 


the length in Part 3 is nonnegative. Therefore, the length of P is greater than or 


equal to w; + cy In view of Equation (12.3), and since Wa = Wy + nq, it is clear 
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that the length of P is at least Wo. This completes the induction argument and 
the algorithm is verified. 
An Example of the Shortest Path Problem with Nonnegative Costs 


Consider the network of Figure 12.4. It is desired to find shortest paths from 
node 1 to all other nodes. Figure 12.5 presents the complete solution for this 
example. The darkened arcs are those used in the selection of the node to be 
added to X at each iteration. These arcs can be used to trace the shortest path 
found from node | to any given node i (how?). As the reader might suspect, it is 
no accident that the darkened arcs form a tree (in fact, an arborescence)! Indeed, 
each time a new node is added to the set_X, it is connected to the nodes in X with 
an accompanying arc leading to this node from some node presently within_X, 
which clearly does not create a cycle. (Notice how the predecessor labels of 
Chapter 9 can be used here with node 1 as the root node.) The tree thus 
generated is known as the shortest path tree, or the shortest path arborescence. 


Complexity of Dijkstra’s Algorithm and a Primal Simplex Interpretation 

As a rough complexity analysis of the foregoing algorithm, observe that the 
number of elementary computations in each of the (m — 1) iterations of the 
algorithm is bounded above by a polynomial of order O(m?). Thus, the 


algorithm has a complexity bound of O(n? ). However, it is computationally 


beneficial to update the calculations of the path lengths to the nodes rather than 
recompute them from scratch at every iteration. That is, whenever a new node is 
added to X, its forward star may be scanned to possibly update any of the 


distance labels w; for the nodes in X. The node having the smallest label w; 
can then be transferred to X, and the current distance calculations for the nodes 


in X can be retained instead of being erased. (Because this smallest label node 
will have its label permanently set and never revised again, we still refer to this 


Figure 12.4. An example of the shortest path problem. 
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procedure as a label-setting algorithm.) In this manner, each iteration is of 
complexity O(m), and the algorithm is of complexity O(m?). Observe also by 


the foregoing inductive validation of the algorithm that at each (nonnull) step of 
the algorithm, the procedure determines a shortest path from node 1 to the 
selected minimal labeled node i. In fact, if at any iteration (< m — 1) of the 


algorithm, it turns out that the arc set (X,X) is empty, then the unlabeled nodes 


in X are all obviously unreachable from node | and the nodes in X have their 
shortest paths from node 1 identified by the current shortest path tree. If all 
nodes are reachable from node 1, the algorithm will find the shortest paths from 
node | to all the nodes at the end of iteration (m — 1). 

It turns out that the foregoing algorithm is actually an implementation of 
the primal simplex method. To see this relationship, assume for convenience 
that all nodes are reachable from node 1. Consider a minimum—cost network 
flow problem in which a supply of (m — 1) is placed at node | and each of the 
other nodes has a demand of one unit (i.e., b} =(m—1) and b, = -1, Vi =2,...,m). 


Suppose that we start with an all artificial basis, by designating node | as the 
root node, and constructing artificial arcs having big—M cost coefficients to 
directly connect node 1 with each of the nodes i = 2,..., m. Then, it is easy to 
verify that the first simplex iteration using the rule of entering the variable 


having the most positive z — cj value is identical to the first iteration of 


Dijkstra’s algorithm. In fact, this is true for all the iterations. To see this, 
suppose that at some stage, we have a set of nodes still connected to node 1 via the 
artificial arcs, while the remaining nodes are connected to node 1 by (directed) 
paths in the current shortest path (sub)tree. Let X be a set that contains the latter 


nodes, and let Y contain the former nodes. Note that the dual variables w, for 


nodes i € ¥ are all —M, while the dual variables w, for the nodes i € X are 


equal to the negative of the total cost on the arcs in the respective paths 
connecting the nodes to node 1 in the current basis tree. Hence, we currently 
have w,; = —w};, as given by Dijkstra’s algorithm. Note that the maximum (2y - 


cj value is realized for an arc (i, j) having i ¢ XY andj € X (why?). Therefore, 


the value 


max {Z;; — 


= max {w,—w,—cy ie X, je X} 
GJ) 


Ci j 


= -min {-M+w} +c, :ie¢ X,j €X} 
Gs) 
is realized by the same arc (p, q) as in Dijkstra’s algorithm (why?). When the 
arc (p, q) enters the basis, the only reverse arc in the resulting cycle is the 
artificial arc, which therefore leaves the basis. Moreover, the subtree of the 
resulting basis tree induced by the nodes in X U {q} is identical to the current 


shortest path tree constructed by Dijkstra’s algorithm. Hence, this algorithm 
coincides with a primal simplex implementation and produces an optimal basis 
tree for the previously—mentioned network flow problem at termination. This is 
verified by the foregoing validation of Dijkstra’s algorithm (why?). 
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A Shortest Path Procedure for Arbitrary Costs 


The shortest path algorithm described earlier in this section does not generalize 
to the case when the costs are allowed to be negative. Figure 12.6 illustrates this 
where the previous algorithm would select node 3 to enter X with w; = w+ ¢3 


= 2 as the value of the shortest path from node | to node 3. However, it would 
be better to first travel to node 2, incurring a higher cost, and then go on to node 
3 for a saving. 

There still is a fast and efficient method for the shortest path problem 
with negative (or mixed-sign) costs. We shall assume, however, that the sum of 
the costs on arcs comprising any circuit in G is nonnegative. Without this 
assumption, a “traveler” would proceed directly to the circuit in G and traverse 
it an infinite number of times with the cost decreasing after each time around the 
circuit. Note that placing an upper bound of unity on the arcs does not help solve 
this problem! (See Exercise 12.30.) Indeed, the problem of finding a shortest 
simple path on networks having mixed-sign costs is NP-complete (see the 
computational complexity references of Chapter 8). 

It is interesting to note a certain property at this point, which appears to 
be intuitively obvious, but is true only in the absence of negative cost circuits. 
Suppose that G has no negative cost circuit, and that a shortest simple path from 
node 1 to some node q is given by A ={l,...,p,q}, where node p is the 


predecessor of node qg on this path. Let the length of the simple path PR, = 
{l,.... p} = A \{q} be L,. Then, Z; is the length of the shortest simple path from 
node | to node p. To see this, suppose that, on the contrary, there exists another 
shorter simple path P; from node | to node p. If g ¢ P;, then we have a 
contradiction to A being a shortest simple path to node q, because this would 
mean that traveling from node 1 to node p as in A and then transitioning to 
node q would yield a shorter simple path than A. On the other hand, ifg € BP, 
then suppose that P, = {1,...,¢,....9}, where the length of the segment from node 
1 to node g in R equals Ly, and the length of the circuit comprised of the 


Figure 12.6. An example where the nonnegative cost algorithm 
will not work. 
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segment of P; from node g to node p, and then back to node q via the arc (p, q), 


equals C. Hence, the length of Pj equals L, +C—c,,, which we are asserting 1s 


Pq? 
lesser than Ly, 1e., Lg +C<L,+c,,. Note that the left-hand-side of this 


inequality is the length of a nonsimple path from node | to node g, whereas the 
right—hand-side equals the length of the shortest simple path A from node | to 
node q, thereby portending the existence of a negative cost circuit. Indeed, since 
1, +Cpgq < Ly because Ly is the length of a particular simple path from node | to 


node g, we have from the previous strict inequality that L, +C < Ly, or that C < 
0, which contradicts our assumption that there does not exist any negative cost 
circuit. By induction, therefore, the shortest simple path from node | to any 
intermediate node in F, is given by the corresponding segment of A from node 
1 to that node. 
As an example of the foregoing discussion, consider a graph 
GO. / 4.) with .7 °° = {1, 2, 3}, .7= {, 2), C1, 3), (2, 3), (G, 2)}, and 
where cj = 2, cj3 = 5, C23 = 1, and cz. =—4. Then it is readily verified that 
the shortest simple path from node | to node 3 is given by {1, 2, 3}, which is of 
length cj + ¢23 = 3, but the shortest simple path from node | to node 2 is not {1, 
2} of length cyy = 2, but rather, it is given by {1, 3, 2} and is of length c,3 +c3) 
= 1. Hence, there must exist a negative cost circuit in this graph; indeed, the 
loop formed by the arcs (2, 3) and (3, 2) is of length c73 +¢3) =—3. 
The algorithm for the present case works with the dual of the shortest 
path problem. Recall that the dual problem with the substitution w; =—w, for i 
= |,...,m is given by the following: 
Maximize w,,-w 
subjectto w,—w;< cy i,j =1,...,m 


w; unrestricted, i=1,...,m. 


Because the objective and the constraints involve only differences in variables, 
we may set one variable to any value, say wj = 0 (why?). 


In the algorithm for mixed—sign costs we shall begin with a choice of w’ 
that is “superoptimal” with respect to the dual objective, but which may violate 
one or more of the dual constraints. We shall show that by iteratively modifying 
w’ to satisfy the constraints, one at a time, we shall be able to terminate in a 
finite number of steps with an optimal solution. 

Considering the set of nodes reachable from node | and assuming these 
nodes to be i = 2,..., m, and noting the dual constraints, the fundamental task is 
to determine a set of labels w; equal to the length of some path from node | to 


node i for i = 2,..., m such that 


w; =minimum {w;+cj,;} for all j= 2,..., m. (12.4) 
i:(i, j) exists 
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If w;, i=2,..., m, are the respective lengths of the shortest (simple) paths from 
node | to nodes 2,..., m, then Equation (12.4) must be satisfied in the absence of 
negative cost circuits. To see this, consider a shortest simple path A = {1,....p, 
J} from node | to node /, where node p is the predecessor of node j on this path. 
Hence, by our previous discussion, the segment {1,..., p} describes a shortest 
simple path to node p, which therefore yields w =w), +c,; 2 minimum 
Ii, 7) exists 


{wj +c} =w, +c, say. If equality holds in the foregoing inequality, then 


(12.4) is satisfied. Hence, suppose on the contrary that we have 


’ ’ 
Wy toy < Wj. 


Now, let P, = {1,..., w} be a shortest simple path from node | to node u, which 
is given to be of length wi. If / ¢ P), then the foregoing strict inequality 
contradicts that w’; is the length of a shortest simple path to node j (why?). On 
the other hand, if je, ie. R = {1,...,/,... wu}, then letting Z denote the length 
of the segment {1,..., 7} in P), and letting C denote the length of the circuit 
comprised of the segment {j,..., u} of P, plus the length Cy of arc (u, j), we have 


by the foregoing strict inequality and the fact that w; < L (why?): 


L+C=wit+c 


' 
uj < ; <L, 


or that C <0, i.e., there exists a negative cost circuit in G, a contradiction. 
Therefore, (12.4) holds true. 

Conversely, if Equation (12.4) is satisfied, then the labels w;, i= 2,..., m, 
must be the lengths of the desired shortest simple paths. If this is not true for 
some node j # 1, say, then consider a shortest simplest path {1,..., 9, 7... /} 
from node | to node j. Note that for each node & in this path, the path from node 
1 to node & in this path must be a shortest simple path to node & (why?). Hence, 
let node r, say, be the first node in this path for which the label wy exceeds the 
length of a shortest simple path from node | to node 7. (Here, node 7 could be 


node / itself.) But from Equation (12.4), we know that w, < w) + cg, = (the 


length of a shortest simple path to g) + c,,, which equals the length of a shortest 


qr? 
simple path from 1 to r, a contradiction. Hence, we have established the 
following key result. 

Theorem 12.2 

Let nodes i = 2,..., m be the nodes reachable from node 1 in a network having no 
negative cost (directed) circuit reachable from node 1. Suppose that w; is the 
length of some path from node 1 to node i for i= 2,..., m. Then w; is the length 


of a shortest path from node | to node i for i = 2,..., m if and only if Equation 
(12.4) holds true. 
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Consequently, our task is to ensure that Equation (12.4) is satisfied. The 
astute reader will have observed the relationship between Theorem 12.2 and the 
(Karush-—Kuhn—Tucker) primal—dual feasibility and complementary slackness con- 
ditions for the shortest path problem. For the case of nonnegative arc costs, we 
were able to satisfy Equation (12.4) by a label-setting procedure in which the 
first time a node was selected, it received a label equal to the length of its shortest 
path from node |. In view of the example in Figure 12.6, such labels may need 
to be revised in the presence of costs having mixed-signs. Hence, the following 
modification of Dijkstra’s algorithm is known as a label-correcting algorithm. 

An algorithm for finding the shortest path distances from node | to all the 
nodes proceeds as follows, assuming that G has no negative cost (directed) 
circuit. Recall that for a node p, the forward star of p is the set of nodes j such 
that (p, /) is an arc in the network. We also follow the convention that a node not 


reachable from node | has a shortest path distance of « and that «0 + ej; = 00 for 


-o< Cy <0. 


INITIALIZATION STEP 
Set w, =O and w; =oo fori # 1. Let the scan eligible list SE = {1}. 


MAIN STEP 


If the scan eligible list SE = ©, stop; the labels w; are the shortest path distances 
from node | to nodes / = 2,..., m. (The corresponding shortest paths are traceable 
via the predecessor labels or the “darkened” arcs.) Otherwise, extract (i.e., select 
and remove) a node p € SE and scan the forward star of p. For all arcs (p, q) 


: . : , , eae | 
corresponding to this set for which w, > w, +Cpg, Set Wg =W), +Cpq, let p be 


the predecessor of node q, and replace SE by SE U {q}. (At the end of the step, 
the “darkened” edges correspond to the arcs connecting each node / that has _w; 
<0 to its predecessor node.) Repeat the main step. 


Figure 12.7. A shortest path example with negative costs. 
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An Example of the Shortest Path Algorithm for Arbitrary Costs 


Consider the network of Figure 12.7 where we wish to find the shortest path 
from node | to all the nodes. In Figure 12.8 we present the complete solution of 
the example by the previous algorithm. There is no required order in which the 
nodes must be selected from SE at each stage for the algorithm to converge. 
However, as we shall see subsequently, the choice of selecting the next node 
from SE does affect both the theoretical and practical efficiency of the 
algorithm. When the algorithm stops, Figure 12.8d gives the shortest paths and 
their values w;, along with the shortest path tree or arborescence. Arcs (i, /) 
along the shortest path (ie., belonging to the shortest path tree) have 


Wi — w= cj. For any are (i, /) in the shortest path tree at any stage, where i is 


the predecessor of j, if we have the property w} = w; +c, holding true, we say 


that the corresponding labels are sharp with respect to this arc. Hence, the 
foregoing statement asserts that the w’—labels are sharp with respect to the 
shortest path tree arcs at termination of the algorithm. We shall provide more 
rigor and insights into this observation in the discussion that follows next. 


Verification of the Algorithm for Arbitrary Costs 


We shall first show that when w; <0, this value represents the cost of some 
path (not necessarily simple) from node 1 to node i. We note that in computing 
the cost of a (not necessarily simple) path, we must count the cost of an arc as 
many times as the arc appears in the path. Thus, for a nonsimple path P = {(1, 
3), (3, 4), (4, 5), (5, 3), (3, 4), (4, 6)} the associated cost would be c)3 +¢34 + 
C45 + C53 +34 +Cgg. As we shall see, nonsimple paths play a part in the algo- 
rithm only when there is a negative cost circuit in G. 

We shall demonstrate that w; (< 0) represents the cost of a path from 
node | to node i at every iteration of the procedure. 

Now, since we start with w; =0 and w; = fori # 1, the result is true at 
the first iteration. This is so because w, = 0 represents the cost of the (empty) 
path, which contains no arcs, from node 1 to node 1. 

Assume that the result is true at iteration ¢; that is, suppose that w; (<0) 
represents the cost of some (not necessarily simple) path from node 1 to node i. 


Consider iteration ¢ + 1. Suppose that for i e SE, we find that w; + cy < w,, in 


which case, we set w} = w; + c,- By assumption, there exists a path F; from 
node | to node j at iteration ¢ having cost w;. Consider the path P= PUGS}. 


This path has a cost of w; + c = w' and the result is true at iteration ¢ + 1. Thus, 


i 
if w, <0, then there exists a (not necessarily simple) path P from node | to 
node k along which YG, jyepcy = Wk- 

If there is no negative cost circuit in G, then the cost of any nonsimple 
path is greater than or equal to the cost of the corresponding simple path after 
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eliminating the circuits in this path (why?). Hence, in the absence of negative 
cost circuits, w; is bounded from below by the cost of a shortest simple path, 
and thus by a finite integer (why?). Finally, treating 00 as a large integer, since 0 
< |SE| < m — 1, and since the two-tuple {X;/, | SE |} lexicographically 


decreases by a positive integer at each iteration (why?), the shortest path 
algorithm will stop in a finite number of steps if G has no negative cost circuit. 
Moreover, at termination, if any w; = 00, then it is readily verified that node i is 


unreachable from node 1, and vice versa. 
Let us show that at termination, the labels w; are the corresponding 


shortest path distances. From Theorem 12.2, it is sufficient to show that 
Equation (12.4) holds true for the nodes reachable from node 1. Consider any 
node q ~ 1, and let node u be the predecessor of node q at termination. Then we 


must have w, = w, +Cyg, i.e., the w’—labels must be sharp for the arc (u, q). To 


see this, note that this is true when w, was finally set at its terminal (minimal) 


value with node wu being its predecessor. If, on the contrary, the label w), was 


subsequently revised (and therefore lowered), then node u would have entered 
SE and, when selected for scanning, would have subsequently revised the label 
of node g to a lower value, which contradicts the final setting of the label of q. 
Hence, we have, 


wi =w +c, > minimum {w;+c,,}=w,+c,., say. 
ie ae oes i +Cigh = Wp +Cpqs SAY 
If equality holds in the foregoing inequality, then (12.4) is satisfied for this 


(arbitrary) node q. Hence, on the contrary, suppose that w, > Ww), +Cpg- 


< co was set for the final time in the algorithm and was later 


, 


Pp 
selected from SE for the final time, we should have relabeled node q since 


Wa > Wp + yg. Hence, we have a contradiction and therefore, the algorithm 


produces the required shortest paths. 

In fact, at termination, the predecessor labels produce a shortest path tree (actually, 
an arborescence). To see this, note that we commence with an arborescence 
having node 1 as the root node being directly connected to each of the nodes 
2,..., m via artificial big—M (virtually, infinite cost) arcs. Inductively, suppose 
that we have an arborescence at any given stage when we scan the forward star 


of some node p € SE, and discover a node q in this set having Wa > Wh + Crags 


based on which, we revise the label w, to equal w,+c,, and we set the 


predecessor of node qg to node p. If node q already had node p as its predecessor, 
this would not alter the current arborescence (this step would only serve the 
purpose of making the labels sharp with respect to the arc (p, q)). Otherwise, 
suppose that we trace the chains (reverse paths) from nodes p and q to the root 
node 1, and let & be the first common node encountered on these chains. First, 
suppose that k # q (we might have & = p or k = 1). In this case, let u # p be the 


However, after w 
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previous predecessor of node q (which belongs to the non-null chain from q to 
k). When we change just the predecessor of node q from u to p, this is equivalent 
to adding the arc (p, q) to the current arborescence and removing the arc (w, q). 
The resulting graph is also an arborescence (why?). Observe that this is akin to 
performing a primal network simplex type tree update. However, it is insightful 
to note at this point that we are not executing a full dual update because we only 
revise the label of node qg alone to the lower value, and do not simultaneously 
decrease by the same amount the labels of all the nodes in the subtree i 


containing qg that is obtained by deleting arc (uw, q) in the current tree, as we 
would in a network simplex iteration. Hence, although the node labels are sharp 
at termination for the arcs in the final tree, they are not necessarily sharp for all 
the arcs in intermediate trees. Hence, unlike as for Dijkstra’s algorithm for the 
nonnegative cost case, the present scheme differs from a standard network 
simplex implementation. 

On the other hand, suppose that q = k. In this case, the current 
arborescence contains a path P = {1,..., u, q,..., p} from node 1 to node p. By 
revising the predecessor label of g from wu to p, 1.e., by adding the arc (p, q) to 
the current arborescence and removing the arc (u, g), we would create a circuit 
comprised of the segment {q,..., p} in the path P, plus the return arc (p, q). Let 
C be the length of this circuit. As one might guess, it must be the case that C < 
0, i.e., we have a negative cost circuit in G, which would contradict our standing 
assumption. To verify this, let L be the length of the segment {q,..., p} in the 
path P, so that C=L + cpg. Note that if the w’ —labels were sharp with respect 


to all the arcs in the path segment {q,..., p} within P, we would have 


w, =w,+L (why?). However, as discussed above, we may not have this 


sharpness property holding true for all the arcs in the path segment {q,..., p} of 
P. Hence, in general, we have Wp > Wy +L> Ww +Cpqg +L, or that C=L+t 
ng < 0, a contradiction. 

Assuming that all nodes are reachable from node 1 (via big—M artificial 
arcs, if necessary), the reader may now observe that the final resulting shortest 
path tree (arborescence) produced is primal feasible for the corresponding 


minimum cost network flow program that has b=m—1 and b; =~, 


Vi =2,...,m. Moreover, the dual solution w; =—wj, i= 1,..., m, is dual feasible 
and is complementary slack with respect to this solution. Hence, this tree yields 
an optimal basis. 

We also have at hand a way to determine whether the network contains a 
negative cost circuit reachable from node 1. If no negative cost circuit exists, 


then cy => cy <0 is a lower bound on the w’ —labels (why?). Thus, if any w; 


falls below cg during the algorithm, a negative cost circuit must exist and we 
can then terminate the shortest path algorithm. 
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A Computational Complexity Remark 


Suppose that we are given a shortest path problem having nonnegative cost 
coefficients. We can use the foregoing procedure for this problem and select at 
each stage that node from the scan eligible list SE, which has the smallest label 
Wh Because cj > 0, all labels revised henceforth will be set at distances greater 


than or equal to Wp. In particular, this means that node p itself will never have 


its label revised again, and so will never re-enter SE. Consequently, the 
algorithm can perform at most m iterations in this case. Noting that each 
iteration is of complexity O(m) because it involves scanning only the forward 


star of the selected node, the algorithm in this case is of complexity O(m’). 


Observe that in such an implementation, we can maintain the scan eligible list 
SE as a heap, where the nodes p < SE are ordered according to nondecreasing w, — 


values. (These latter values are referred to as the key for maintaining the heap in 
this context, with the associated function key(p) = Wp-) Then, at each stage, we 


extract (i.e., select and delete) the node from the top of the heap SE (which 
simply takes O(1) time). Also, whenever a new node is introduced into SE, it is 
inserted within this list such that we preserve the nondecreasing order of the 
corresponding key values. This so-called insert operation can be performed in 
O(log(m)) time. We refer to this enhanced implementation of Dijkstra’s basic 
algorithm as the Heap—Dijkstra procedure. Note that, in particular, if we are 
simply interested in finding a shortest path from node 1 to some specific 
terminus node f¢, then the Heap—Dijkstra procedure can be terminated with an 
optimal solution value w, at the stage when node f is extracted from the heap SE 
(why?). 

It is important to note that, for the arbitrary cost case, some caution needs 
to be exercised to ensure a polynomial-time algorithm. For example, a first—in— 
first-out scheme for selecting nodes from SE yields a polynomial-time 
algorithm, as shown in the following section, while a last—in—first—out scheme is 
known to admit an exponential computing effort. 


A Labeling Algorithm for the Shortest Path Problem 


Either for hand or computer calculations there are simple and convenient ways 
to maintain the information required to solve a shortest path problem having 
arbitrary costs. Suppose that we associate with each node j a label L(/) = (i, wi) 


containing two pieces of information. The second entry, Wi, in L(/) indicates the 
cost (length) of the current “best” path from node | to node j. The first entry, i, 
in L(/) gives the predecessor node, that is, the node just prior to node j in the 
path. Let cg = dey <0 ¢j- The labeling algorithm is as follows: 
INITIALIZATION STEP 


Set L(1) = (-, 0), L(i) = (, ©) for i=2,..., m, and SE = {1}. 
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MAIN STEP 


If SE=, stop; the required shortest paths are available. The second label on 
the nodes gives the shortest path distances and the paths are traceable using the 
first (predecessor) labels. Otherwise, select p € SE. For all arcs (p, q) with g in 


, , = ye 
the forward star of p such that Wa > Wy tlpq, Set L(g) = {P, Wy = Wp + Enq}; 


and replace SE by SEU{q}. If Wy <cg, stop; there is a negative cost circuit in 
G. Otherwise, remove p from SE and repeat the main step. 
An Example of the Labeling Algorithm 
Suppose that we use the labeling algorithm to solve the shortest path problem of 
Figure 12.7. First, cg =-1-4-6=-ll. 

The sequence of operations of the labeling algorithms are as follows: 
L(1) 7 G, 0), 12) ~ , °°), L(3) = C, oo), (4) = (-; 00), SE= { l }. 
p= l, (2) 7m qd, 2), L(3) z= ad, -1), SE= {2, 3}. 
p= 2, L(3) = (2, -2), L(4) = (2, 5), SE= {3, 4}. 
p- 3, (4) - (3, -8), SE= {4}. 
p=4,SE=©. 
STOP; 1(2) = (1, 2), L(3) = (2, -2), and £(4) = G, —8). The shortest paths to 
nodes 2, 3, and 4 are of lengths 2, —-2, and -8, respectively, and are respectively 
determined as {(1, 2)}, {(1, 2), (2, 3)}, and {(1, 2), (2, 3), (3, 4)} by backtrack- 
ing using the first labels in L(-). 


Identifying a Negative Cost Circuit With the Shortest Path Algorithm 


Cie to 


We have already indicated that if w, <cg at some point in the shortest path 


algorithm, then a negative cost circuit reachable from node | exists in G. To find 
such a negative cost circuit, begin at node & and apply the following procedure: 


INITIALIZATION STEP 
Let p =k, C= {k}. 
MAIN STEP 
Let ZL,(p) be the first entry in L(p). 
1. IfL(p) > 0, let 2= L,(p), and replace L,(p) by —L,(p), and C by C 
U{2}. Set p = @ and repeat the main step. 


2. If L(p) <0, then stop. A negative cost circuit has been found, and 


may be traced by starting at the last element in C and proceeding 
backwards through C until this element repeats. 


Note that the original node & may not be part of the negative cost circuit. 
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12.3 POLYNOMIAL-TIME SHORTEST PATH ALGORITHMS FOR 
NETWORKS HAVING ARBITRARY COSTS 


We will now slightly modify the algorithm for determining shortest paths from 
node | to all the other nodes in a network having arbitrary cost coefficients in 
order to derive an efficient polynomial-time (label—correcting) algorithm. The 
particular algorithm we discuss is known as the partitioned shortest path (PSP) 
algorithm, because it effectively partitions the scan eligible list SE into two sets, 
namely, NOW and NEXT. As implied by the names of these sets, the nodes in 
the list NOW are scanned first. This scanning process postpones the 
consideration of any mew scan eligible nodes into the list NEXT until NOW is 
exhausted. At this point, if NEXT is nonempty, its contents are transferred to 
NOW and the procedure repeats. This continues until SE = NOW VU NEXT is 
empty. Notice that the first—in—first-out (FIFO) scheme for selecting a node 
from SE is a particular case of this procedure, where nodes are selected from the 
top of NOW for scanning, and any new node added to NEXT is inserted at the 
bottom of this list. We shall have more to say about this strategy later. The 
algorithm is specified next. 


Partitioned Shortest Path Algorithm 


INITIALIZATION 


Set wj =0, w; =o fori # 1, NOW = {1}, NEXT = ©, and the iteration counter 
k= 1. Let cg be the sum of the negative costs in G. 


STEP 1 


If NOW = ©, proceed to Step 2. Otherwise, extract (select and delete) a node p 
€ NOW. For each arc (p, g) with g in the forward star of p such that 


, , y , 
Wa > Wy tCpg Set Wy = Wy + Cyy and let p be the predecessor of node q. If 
, 


W, <co for any such node, stop; there exists a negative cost circuit in G. (This 


circuit can be traced as discussed above.) If node q is not currently scan eligible, 
that is, it is in neither NOW nor NEXT, add it to NEXT. Repeat Step 1. 


STEP 2 


If NEXT = ©, then stop; the labels w; give the shortest path distances. The 


shortest paths are traceable by backtracking using the predecessor labels. Other- 
wise, put NOW = NEXT and NEXT = © (perhaps by simply renaming the 
lists), increment & by one, and return to Step 1. 


Example of the PSP Algorithm 


To illustrate, consider the example of Figure 12.7. As before, let Z(-, -) be the 
predecessor and path length label as defined in the previous section. We begin at 
iteration A= 1 with NOW = {1}, NEXT = ©, L(1) =(, 0), and Ld = (-, -) for 
i# 1. The operations proceed as follows: 
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1. k=1 with NOW = {1}, NEXT=@: 
p= 1,L(2)F(/, 2), L3)=(1, -1), NOW = ©, NEXT = {2, 3}. 
2. k=2 with NOW = {2, 3}, NEXT=®: 
p=2, L(3)=(2, -2), L(4)=(2, 5), NOW = {3}, NEXT = {4}. 
p= 3, L(4)=(G, -8), NOW = @, NEXT = {4}. 
3. k=3 with NOW = {4}, NEXT =@: 
p=4, NOW =@, NEXT =@. 


The algorithm terminates with Z(2) = (1, 2), Z(3) = (2, -2), and L(4) = (3, 
—8). These labels give the shortest paths from node | to nodes 2, 3, and 4 as 
before. 


Complexity Analysis of the PSP Algorithm 


Assume that the graph G does not contain any negative cost circuit (reachable 
from node 1). First of all, note that the algorithm terminates finitely and pro- 
duces the shortest paths from node 1 to all the other nodes that are reachable 
from it. This follows from the previous section by simply noting that SE = NOW 
U NEXT and that the algorithm only specifies a (partial) order in which the 
nodes in SE must be considered. However, it is this partitioning of SE that 
guarantees a polynomial-time algorithm. As we now verify, the algorithm 
performs at most (m — 1) iterations. Because each iteration scans no arc of the 
problem more than once, it is of complexity O(|.-/|), where |.”| is the 


number of arcs. Hence, the algorithm is of complexity O(m| -Y |). 


The key property of the algorithm is as follows. Consider the current 
(shortest path) partial tree that exists at the beginning of Step 2 at each iteration 
k in the algorithm. Then the /evel of each node in NEXT in this tree is at least k, 
where the level of a node is the number of arcs in the chain from this node to 
node | in the tree. This is clearly true for k = 1. Inductively, assume that this is 
true for iteration k and consider iteration (k + 1). Note that at the beginning of 
iteration (k + 1), the list NOW coincides with the list NEXT at Step 2 of the 
previous iteration. Hence, by the induction hypothesis, each of the nodes in this 
list has a level of at least & in the tree. A node that has its label revised in this 
iteration is either already in NOW or enters NEXT, but with its level in the tree 
being one more than the level of the node that revises its label. Hence, the levels 
of the nodes in NEXT at the end of Step | (or the beginning of Step 2) of 
iteration (k + 1) are at least (k + 1), thus establishing the required result. 

Because of this key property, since a shortest path tree is constructed by 
the algorithm in the absence of negative cost circuits in G, the shortest paths 
from node | to the nodes that are in NEXT at the beginning of Step 2 of each 
iteration k involve at least k arcs. This follows since the shortest path to any 
node is verified or established in the iteration following the one in which it 
shows up in NEXT at Step 2 for the last time. This means that the algorithm 
cannot possibly perform more than (m — 1) iterations (why?). (Indeed, if a node 
has its label revised at iteration m (or higher), it must lie on a negative cost 
circuit, which can be traced as discussed previously.) In fact, some node in 
NEXT at Step 2 must have its shortest path already determined at each iteration. 
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This follows because the current list NEXT becomes the list NOW for the next 
iteration, and since all nodes relabeled from this point onward must have a node 
in this list on the chain to node 1, we cannot possibly have all the nodes in this 
list repeating or else we would have a cycle. 

For the foregoing example, at the end of iteration A = 1, node 2 ¢ NEXT 
has its shortest path determined, and at the end of iteration k = 2, node 4 € 
NEXT has its shortest path determined. Node 3 shows up in NEXT for the last 
time at (the end of) iteration A = 1 and has its shortest path determined during 
iteration k = 2. The levels of nodes 2, 3, and 4 in the final shortest path tree are 
1, 2, and 3, respectively. 

As far as computational implementation strategies are concerned, note 
that we could maintain the list NEXT as a heap, where the nodes are arranged in 
nondecreasing order of the labels w; (which serve as the key for the heap). 


Hence, each time a new node q is introduced into NEXT (or the label of an 
existing node q is revised within NEXT), it is inserted (or re-inserted) into 
NEXT to preserve the nondecreasing order of the labels w;. Then, once NEXT 


is transferred to NOW at the end of any iteration, the nodes are subsequently 
extracted one at a time from NOW from the top of the list. However, in spite of 
its efficiency, this strategy turns out to be computationally expensive relative to 
a simple FIFO procedure because of the relative ease with which shortest path 
problems are solved using Algorithm PSP. On the other hand, a slight 
modification to the FIFO strategy yields an empirical advantage. In this strategy, 


letting Wivop be the label of the node that is currently at the top of the list 


NEXT, whenever a new node g having the label w is inserted into NEXT, then 


this node q is added to the top of the list NEXT if w, < w), 

0} 
is inserted to the bottom of this list. The following section discusses a somewhat 
more sophisticated variation of this concept of maintaining an approximate heap 


that turns out to be even more effective in practice. 


Threshold Partitioned Shortest Path Algorithm 


, and otherwise, it 
Pp 


A noteworthy computationally efficient variant of the PSP algorithm, particu- 
larly for nonnegative cost coefficients, has been designed to operate as follows. 
At the end of every iteration k, a suitable threshold value t is computed. Instead 
of transferring the entire list NEXT into NOW, only those nodes 7 € NEXT are 
placed in NOW that have w; <7. The value ¢ is tuned heuristically based on the 
network structure, but of course always satisfies ¢ > minimum {w;: i € NEXT}. 
Additionally, at Step 1 of the PSP algorithm, whenever a node q has its label 
updated and q is not a member of NOW, it is checked whether or not the revised 


w, exceeds the current threshold value 7. If w, > ¢ then node q is added to 


NEXT as before. Otherwise, node g is added to a separate list NOW’ (unless it 
is already there) and it is removed from NEXT if it happens to be there. Then, 
when NOW becomes empty, the contents of NOW’ are transferred to NOW and 
the process repeats by scanning the nodes in NOW as before, until both NOW 
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and NOW’ are empty. At this point, if we also have NEXT = ©, then the 
method terminates. Otherwise, an appropriate revised threshold value f¢ is 
computed, and all nodes i in NEXT that have their label values w; < ¢ are 
transferred to NOW, while the remaining nodes are retained in the list NEXT. 
The algorithm then resumes processing the nodes in NOW as before. 

Observe that when all costs are nonnegative, the nodes having the small- 
est w;—labels in the list NOW’ or NEXT that are transferred into NOW have 
their shortest path distances already determined, and so will not show up in any 
subsequent set. Hence, there can be no more than (m — 1) ‘otal transfers from 
NOW’ or NEXT into NOW. Because the effort involved between each such 


transfer is of complexity O(|.*/ |), the algorithm is of complexity O(m| .% |). 


Computationally, any new nodes added to NOW’ or NEXT can either be 
appended to the bottom of these lists, or can be inserted into these lists as 
suggested for Algorithm PSP. Nodes are typically extracted from the top of the 
list NOW for scanning next, but it has also been suggested that nodes from 
NOW be selected in a last—in—first-out order. Empirically, the threshold 
partitioned shortest path algorithm has been found to be computationally 
superior on several types of networks, particularly, having nonnegative costs 
(see the Notes and References section). 


Some Extensions of Shortest Path Problems 


We conclude this section by indicating several interesting variants of shortest 
path problems that have been analyzed in the literature. One such variant is the 
time-dependent shortest path problem, which frequently arises in transportation 
networks that are subject to traffic congestion. Here, the duration required to 
traverse any arc (the arc cost) can be time-dependent, i.e., in general, it can be a 
function of the time that this arc is entered via its from—node. One can concep- 
tualize such networks in an expanded time-space representation in which each 
node i is replicated for all possible times ¢ that it can be entered, and accord- 
ingly, it is represented by the two-tuple (i, 4). An arc connects node (i,1;) to a 


node (j, f,) in this network whenever there exists an arc (i, /) in the original 
network for which the time to traverse this arc when entering node j at time 4 
equals (¢, —4). The “cost” on this arc is therefore simply a constant equal to 
(t2 —t,). By conceptualizing the problem over this time-space network either 


explicitly or implicitly, depending on the nature of the traversal time functions, 
the various algorithms discussed for the standard shortest path problem can be 
extended to such time-dependent problems. In particular, if the arc traversal 
times possess a first—in—first—-out property, i.e., entering an arc at a relatively 
earlier time ensures exiting it also at a relatively earlier time, then this time— 
dependent shortest path problem is solvable in polynomial time as for the 
standard case. Otherwise, in general, the problem is known to be NP-hard (see 
the Notes and References section for definitions and details). Further extensions 
of this problem have also been considered in which arcs additionally have par- 
ticular labels (e.g., designating modes of transportation), and one is required to 
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determine a time—dependent shortest path for which the corresponding sequence 
of labels belongs to a set of acceptable strings of labels. For instance, in a mul- 
timodal network having various modes of transportation such as cars, buses, 
trains, and even walking or biking, a user might specify various admissible 
strings such as, e.g., walk — train — bus — walk, from an origin to a destina- 
tion. Here, each specified mode can be repeated several consecutive times via 
the arc labels in the prescribed shortest path solution. This is known as a label 
constrained (time-dependent) shortest path problem. Other extensions include 
problems where the traversal time on any arc (i, j) might also depend on the 
previous arc leading to node i (in order to represent, e.g., left turns versus 
straight crossings into arc (i, /) at traffic road intersections), or the traversal 
times might be random variables. The former class of problems are called 
approach—dependent shortest path problems (and can also be label—constrained 
and time-dependent), whereas the latter class of problems are known as 
stochastic shortest path problems, or as dynamic stochastic shortest path 
problems in the case when the actual traversal cost on an arc is (stochastically) 
realized when reaching the entering node of the particular arc. We refer the 
reader to the Notes and References section for further details and information 
regarding these problems. 


12.4 MULTICOMMODITY FLOWS 


In all of the flow problems we have considered to this point, it has not been 
necessary to distinguish among the units flowing in the network. This class of 
network flow problems is called single-commodity flow problems. There is also a 
class of network flow problems called multicommodity flow problems in which 
it is necessary to distinguish among the flows in the network. 

The most natural example of multicommodity flows occurs in rush hour 
traffic in any metropolitan city. If the area is divided into zones, then there are a 
number of people in zone i who must travel to work in zone j. There are also a 
number of people who must travel from zone j to work in zone i. The locations 
at which people originate inherit a corresponding supply (6 > 0), and where they 
wish to go, we obtain a corresponding demand (6 < 0). If we treat the problem 
as a single-commodity flow problem, a minimal—cost flow procedure (network 
simplex or out—of-kilter) would use the supply of people in a given zone to 
satisfy the demand in the same zone. This is an unacceptable solution. In this 
problem and ones like it, we must distinguish between the different types of 
flow and be careful to retain their identity and flow pattern throughout the 
optimization procedure. That is, we must essentially have a different flow vector 
and a set of conservation equations for each commodity. 

As we shall see, multicommodity flow problems do not enjoy the same 
special properties as single-commodity flow problems. As an example, consider 
the network of Figure 12.9. Suppose that there are three commodities that flow 
through the network. The source for commodity 1 is node 1, and the sink for 
commodity | is node 3. That is, commodity | must originate only at node | and 
terminate only at node 3. Similarly, let the source and sink for commodity 2 be 
nodes 2 and 1, respectively. Finally, the source and sink for commodity 3 are 
nodes 3 and 2, respectively. With the restriction that the sum of all commodities 
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flowing on an arc should not exceed the arc capacity Uy = 1, what is the maxi- 


mal sum of commodity flows, /, + /> +/;, possible in the network? 


Finding the maximal flow for the three-commodity problem of Figure 
12.9 is relatively simple since there is only one path that each commodity can 
take on its way from its source to its sink. The paths for commodities 1, 2, and 
3, respectively, are 


A = {(, 2), (2, 3)} 
P, a {(2, 3), (3, 1)} 
P; = {(3, 1), C, 2)}. 


If we place a single unit of flow on any one of the paths, then the other paths are 
completely blocked (that is, must have zero flow), and thus the total flow would 
be |. However, there is a better solution available if we do not require integer 
flows. Suppose that we place 1/2 unit of flow of commodity 1 on A, 1/2 unit of 


flow of commodity 2 on P;, and 1/2 unit of flow of commodity 3 on P,. In this case, 


none of the arc capacities are violated and the total flow of all commodities is 
3/2. From this we see that multicommodity flow problems do not necessarily 
provide integer flows. 

Even though multicommodity flow problems do not have as “nice” a 
structure as single-commodity flow problems, they still are linear programs (if 
we ignore integrality of the variables). As we shall soon see, multicommodity 
flow problems do have a special structure that permits the application of decom- 
position techniques. 


The Multicommodity Minimal—Cost Flow Problem 


Suppose that we are given a network G having m nodes and v arcs in which 
there will flow ¢ different commodities. Let u; represent the vector of upper 


limits on the flow for commodity i in the arcs of the network. Thus, Uing is the 


upper limit on flow of commodity i in arc (p, q). Also, let u represent the vector 
of upper limits on the sum of all commodities flowing in the arcs of the network. 


é ‘ 


to D 
Figure 12.9. A three-commodity maximal flow problem. 
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Then, w,,, is the upper limit on the sum of all commodity flows in arc (p, q). Let 


¢; represent the vector of arc costs in the network for commodity i. Thus, cpg is 


the unit cost of commodity i on arc (p, q). Finally, let b; represent the vector of 


supplies (or demands) of commodity i in the network. Therefore, big is the 
supply (if b;, > 0) or demand (if 5, < 0) of commodity i at node q. 


The linear programming formulation for the multicommodity minimal-cost 
flow problem is as follows: 


t 
Minimize )> ¢;x; 
i=l 
; t 
subjectto > x; <u 
i=l 


where x; is the vector of flows of commodity i in the network and A is the node-arc 
incidence matrix of the graph. The foregoing formulation is called the node-arc 
formulation for the multicommodity flow problem, since it uses the node—arc 
incidence matrix. 

The multicommodity minimal—cost flow problem possesses the block 
diagonal structure discussed in Section 7.5. Therefore, we may apply the block 
diagonal decomposition technique to the foregoing problem. The multicom- 
modity minimal—cost flow problem has (¢ + 1)n variables and n + mt constraints 
(including the slack variables for the coupling constraints and ignoring the 
nonnegativity and upper bound constraints 0 < x; < u;). Thus, even for moderate— 
sized problems, the constraint matrix will be large. For example, suppose that 
we have a problem having 100 nodes, 250 arcs, and 10 commodities. This 
problem will have 2750 variables and 1250 constraints. 

Consider the application of the decomposition algorithm to the minimal-cost 
multicommodity flow problem. Let X; = {x; : Ax; =b;, 0 <x, <u;}. Assume 


that each component of u; is finite so that X; is bounded (see Exercise 12.59 for 
a relaxation of this assumption). Then any x; can be expressed as a convex com- 


bination of the extreme points of X; as follows: 


k; 
xX; = > AyXij 
j=l 


where 
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and Xj, X;25---»Xjz,» are the extreme points of X;. Substituting for x; in the multi- 


commodity minimal-—cost flow problem and denoting the vector of slacks by s, 
we get the following: 


t k; 
Minimize > > (ex, )Ay 
i=l j=l 


t k; 
subject to XL xjAy+s=u 
i=l j=l 


k; 
xX Ay = > i=l, of 
j=} 
Ay 20, Jed kite st 


Suppose that we have a basic feasible solution to the multicommodity minimal-— 
cost flow problem in terms of the A,j—variables, and let (w, a) be the vector of 


dual variables corresponding to the basic feasible solution (w has n components 
and a has ¢ components). Then dual feasibility is given by the following two 
conditions: 

(i) Wpg <0 corresponding to each s,,, and 

(ii) wx, +@; —¢;Xj <0 corresponding to each Ay. 


If any of these conditions is violated, the corresponding variable (s pq OF Ay) isa 


candidate to enter the master basis. Here, s,, is a candidate to enter the basis if 


Pq 


Wpg > 0. For a given commodity i, a nonbasic A,;—variable is enterable into the 


basis if the optimal objective value of the following subproblem is positive 
(why?): 


Maximize (W-—c¢;)X;+@; 
subject to Ax; =b; 
0< XxX; < u;. 


But, since A is a node—arc incidence matrix, this is simply a single-commodity 
network flow problem. Thus, it may be solved by one of the efficient techniques 
for solving single-commodity network flow problems. 


Summary of the Decomposition Algorithm Applied to the Multicommodity 
Minimal—Cost Flow Problem 


We now specialize the decomposition algorithm of Chapter 7 to the multicom- 
modity minimal-—cost flow problem. 
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INITIALIZATION STEP 
Begin with a basic feasible solution to the master problem. Store ae b = 


Bi(t), and (w,a) =é,B!, where ¢; =¢;x, for the A,;—variables. (The 


1 i i 
two-phase or the big—M method may be required.) 
MAIN STEP 


1. Let (w,@) be the vector of dual variables corresponding to the 


current basic feasible solution to the master problem. If any w,, > 0, 
then the corresponding variable s/,, is a candidate to enter the master 


basis. If w,, < 0 for each arc, consider the following ith subproblem: 


Maximize (w-c¢;)x;+@; 


subject to Ax; =b; 
O<x; <u;. 


This is a single-commodity flow problem. If the solution x, _ to this 
problem has z;, —cj, = (w—¢;)x;, +a; > 0, then Ay is a candidate to 
enter the master basis. 


2. If there is no candidate to enter the master basis, then stop; an 
optimal solution is at hand. Otherwise, select a candidate variable, 


: ; -1fe Zi 

update its column according to B a for sp, and B le 
i 

for 4, and pivot. [Note that e,, is a unit vector with the | in the 


row associated with arc (p, q).] This updates the basis inverse, the 
dual variables, and the right-hand-—side. Return to Step 1}. 


An Example of the Multicommodity Minimal—Cost Flow Algorithm 


Consider the two—commodity minimal—cost flow problem whose data are given 
in Figure 12.10. 

The constraint matrix and the right-hand—side are displayed in Figure 
12.11 (the lower and upper bound constraints 0 < x; < uy and 0< x», < uy are 
not displayed). Notice the structure of the coupling constraints and the special 
structured block diagonal constraints. Also, note that x, and x2 represent the 


artificial variables for the two commodities. 


INITIALIZATION 


To avoid the two-phase or the big—M methods, suppose that we begin with the 
following feasible solutions: 
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2,0 3 
ay (4, 3, 4, 5, -2) 7 


(Ugg 2M pqo2pq°"1pq°2,pq) 
(al, 5, 4, 1, -l) (3, 4, 2; 1, 0) 


(a, 1b) 


(7, 5, 3, 0, 2) 


(-2, 0) (-1, 3) 


Figure 12.10. A two—-commodity minimal-—cost flow problem. 


112 2 X212 0 
X123 3 X23 0 
X=} X34 [=] 2 and = X91 =| X34 [=| 3 |. 
141 0 X41 0 
142 0 242 3 


Note that the master basis (in the space of the slack and the 4,,—variables 


consists of all the slacks, 2,,, and A,,. The basis and its inverse are: 


S12 S893, 834 Sy Sqn Ay 

1 0 0 0 0 2 0 

0 1 0 0 0 3 0 

0 0 1 0 0 2 3 

B=| 0 0 0 1 0 0 0 
0 0 0 0 1 0 3 

0 0 0 0 0 1 0 

0 0 0 0 0 0 1 
1 0 0 0 0 —2 0 
0 1 0 0 0 -3 0 
i 0 0 1 0 0 —2 -3 
B =| 0 0 0 1 0 0 0 
0 0 0 0 1 0 -3 
0 0 0 0 0 1 0 
0 0 0 0 0 0 1 


Here, ¢,x,; = 13 and ex; = 24. Denoting | i | by b, we have 


(w,a@) =¢,B ! =(0, 0,0, 0,0, 13, 24) B7! = (0, 0, 0, 0, 0, 13, 24) 
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SECOND COMMODITY 
VARIABLES SLACK VARIABLES 
X34 M41 Man X12 093X034 X41 X04 X—S102.—S893 S834 Sq) Sgg_~—sRHS 


FIRST COMMODITY 
VARIABLES 


*123 


*112 


Coupling 
constraints 
incidence 
matrix for 
Subproblem | 
Node-arc 
incidence 
matrix for 
Subproblem 2 


Node-arc 
Figure 12.11. The constraint matrix for the two-commodity network flow problem of Figure 12.10. 
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4 2 
3 0 
ilu : 7 2 
xX, =B b=B 1 |{=B} 1 j=] 1 
1 5 2 
1 1 
1 1 
z=6¢,B 'b =37. 
Setting up the revised simplex array: 
for the master problem, we get the following: 
Win 93 Way War CRS 


1 
0 
0 
0 
0 
0 
0 


Iteration 1 


First, all w,, <0. Next, we check whether a candidate from either subproblem 


Pq 
(or commodity) is eligible to enter the master basis. 


SUBPROBLEM 1 


w-—c, =0—¢, =(-5,—1,0,—1,-4). 


Subproblem 1 is the single-commodity flow problem defined in Figure 
12.12. An optimal (maximal) solution is x, = (2,3,2,0,0) and the value of the 
Subproblem 1 objective is 


212 —C12 = (W—-¢)) X12 +Qy =-134+13=0. 


Thus, there is no candidate from Subproblem 1. 
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(Maximization) 


(4, -1) 


-2 -1 
Figure 12.12, Subproblem 1 at the first iteration. 
by 
q ,, 0 & 


(Maximization) 


(2, 0) 


(3, -2) 


Figure 12.13. Subproblem 2 at the first iteration. 
SUBPROBLEM 2 
W-—c> =0-c> =(2,0,-2,1,-6). 


Subproblem 2 is the single-commodity flow problem defined in Figure 12.13. 
An optima! (maximal) solution is x77 = (3,0, 3,3,0)' and 
2772 — C72 = (W —C)x99 +Qy = 34+ 24=27. 


Thus, 4’) is a candidate to enter the basis. The updated column for A,» 
(exclusive of 277 —C 2) is 


2 
B!| 0 |=(,0,0,3,-3,0,)!. 


The pivoting process is as follows: 


W223 34 Wa Han a RHS Ang 
; 

re es as er ae 3 
|, 0. Ni 0S 8 0 
ee ck ae ee 0 
sy | 0 O O 1 (3) 
ea Oe ee ee =3 
a4, | 0 0 O 0 0 
Ay |, O20. 20 <8 1 
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Wi2 W337 W3q QD 
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cooocoocoo oe ]o 
cooocoocor o]/o 
cooooroce|/o 

Be 
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Iteration 2 
Again, all Wyq © 0, and sono s ng is a candidate to enter the master basis. 
SUBPROBLEM 1 

(w—c,) = (—5,-1,0,-10, -4). 


Subproblem 1 is the single-commodity flow problem defined in Figure 12.14. 
An optimal solution is x,3 = (2,3,2,0,0)' with 


213 ~Cj3 = (w—¢, )X13 +Q) = -13+13=0. 
Thus, there is no candidate from Subproblem 1. 


SUBPROBLEM 2 


(w ; i) ) = (2, 0, -2, -8, —6). 
Subproblem 2 is the single-commodity flow problem defined in Figure 12.15. 
An optimal solution is x73 = (3, 0,3, 3,0)’ with 
293 — C23 = (w — C7 )X73 +A = —24+24=0. 


Thus, there is no candidate from Subproblem 2. 
Therefore, we already have an optimal solution as follows: 


z = 28 
x} = Ay 1X} i= (2,3, 25 0, 0)' 
X) = Ay Xp) + Aq9X09 


= (2/3)(0,0,3,0,3)' + (1/3)(3,0,3, 3,0) 
= (1,0,3,1,2)'. 
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1 
GB, -5) 
(2) (Maximization) 


(5, -10) (4,-1) 


-2 : =I 
Figure 12.14. Subproblem 1 at the second iteration. 


12.5 CHARACTERIZATION OF A BASIS FOR THE 
MULTICOMMODITY MINIMAL-COST FLOW PROBLEM 


Suppose that we proceed to apply the simplex method directly to the multicom- 
modity minimal—cost flow problem. We first note from Chapter 9 that the 
system Ax; = b; has rank m — | so that it is necessary to add an artificial vari- 


able for each commodity. Adding this artificial column vector, the overall 
constraint matrix is given by 


X; X9 x, xX 

a 1 YT I }n rows 
iA 0 - 0 OO}; }mrows 
‘0 A + 0 0 | } m rows 
|: - i tt 

0 0 A 0) }mrows 


where A =[A,e,,] and I =[I,0]. Selecting a basis submatrix from this matrix, 
we get 


fog 0 


-3 


(Maximization) 


(Uy 54° pq —Cyyq) 


(2, 0) 


(3, -2) 


Figure 12.15. Subproblem 2 at the second iteration. 
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i Ey Ey E, Ei 
A; 0 0 0 
B= 0 A> 0 0 
he 93 : 
0 0 A, 0 


where E; and E are matrices formed by taking selected columns of T and I, 
respectively. The row location of the 1 for a particular column of E; identifies 
the arc used in Aj. 


Because B is a basis matrix, each A; must contain a submatrix that spans 


R™. Therefore, each A; contains an m x m basis (why?). Let us partition A; into 
[B; |D,;] where B, is a basis matrix for Ax; =b;. Note that B; must contain the 
artificial column (why?). From Chapter 9, since B, is a basis for a set of single— 
commodity flow conservation constraints, B; must correspond to a rooted 
spanning tree in G with the artificial variable as the root. Similarly, let us parti- 
tion E; into [E; | E/]. Substituting into B and rearranging the columns, we get 


Cn Eo eo ie E 
BO. 0 DO. OO 
0 0 .. B 0 0 .. D, 0| 


In other words, the basis matrix B has the following general structure: 


, [E' E" E 
6-5 D al 


Denoting the right-hand-side ; by b (where b is a column vector consisting 


of b;,b5,..., and b,), the basic system Bx, =b reduces to 


BE EF fu 
B D 0 D\" lb 
SB 


XF 
where xz is decomposed into | xp |. This system is not particularly easy to 
SB 
solve. However, by utilizing the following change of variables, the structure in 
the system can be exploited, as we shall outline shortly: 
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Xp 1 -B'D 0|[ xp 
xp |=|0 I 0] xy |. (12.5) 
SB 0 0 Iii sz 


This is a nonsingular transformation, and thus we have an equivalent system 
with which to work. On substituting for xp in Bxz =b, we get 


-l ' 

Bs E’ Elo cee ; ap [2] 
D {7 > 

BP Clo oF Ils, 


E E’-EB"D E]| °F |_fu 
i.e., = Xp =| i 
B 0 0 || b 
B 


Now, the second set of equations Bx; = b is easy to solve, since it corresponds 


to a set of rooted spanning trees, one for each commodity. 
Consider the first set of equations in the transformed system after having 
solved for xr: 


” ed | x'p tor 
[E’-E'B DEI, |-o-exp, 
SB 
The solution is 


Xp a ” , -1 -l : el 
” |=[E"-E'BD,E]  (u-E’x’,) 
SB 


and requires the inversion of the matrix [E”- E'B™'D,E]. While this matrix is 


not as easy to invert as B, it is easy to form and understand. First, E is a matrix 
of unit columns corresponding to the slack variables in the basis. Second, let us 


investigate the matrix E”—E'B 'D. 

A typical column of E”—E’B™'D corresponding to commodity k is given 
by e;; -E,B;'a;, where ej; is a unit vector in R” having a 1 in the row corre- 
sponding to arc (i, 7), and aj; is a vector in R”™ having a 1 in row i and a—1 in 
row j. From Chapter 9, recall that y;; = By! a; corresponds to a chain from node 
i to node j in the basis tree. Note that the coefficients of y;j actually reorient the 


chain into a path. Then, -B;\aj corresponds to a chain from j to i in the rooted 
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spanning tree for commodity &. Each coefficient in -B;'a, corresponds to a 
basic variable in B,. Now, Ej is an n x m matrix with its columns being unit 


vectors in R” that identify the basic variables in B,. Thus, -E;,By aj simply 
expands the m—vector -B;' a, to an n—vector by assigning zero coefficients 


corresponding to all nonbasic arcs of commodity k. Hence, —-E;,B;'a; is an n—- 
vector corresponding to the chain from node j to node i in the rooted spanning 
tree for commodity k. Finally, ej —E;,B;'a, corresponds to the unique cycle 
formed when the arc (i, /) is added to the basis tree (and the basic arcs are 


properly oriented). Thus, knowing B, it is easy to form E” — E'B 'D. 
The important conclusion is the following: 


Theorem 12.3 


A transformed basis matrix for the multicommodity minimal-—cost flow problem 
corresponds to a rooted spanning tree for each commodity plus a set of cycles 
and slack arcs. 


Once [E”—E'B 'D, E] is formed as described earlier, we can solve for 
Xp and siz. With the vector 


now known, we can solve for the basic variables x, xp, and $g from Equation 


(12.5). In Exercise 12.61 we ask the reader to develop a systematic procedure 
for computing the dual variables, updating the column of the entering variable, 
and the basis inverse. This coupled with the foregoing procedure for computing 
the basic variables, represents a direct application of the simplex method for 
solving multicommodity flow problems. 


An Example of a Basis Matrix for the Multicommodity 
Minimal—Cost Flow Problem 


Consider the multicommodity minimal—cost flow problem of Figure 12.10 
without the upper bound constraints on the individual commodities. Recall that 
the constraint matrix is shown in Figure 12.11. 

Suppose that we select the basis submatrix (the reader is asked to verify 
that this is a basis submatrix) indicated in Figure 12.16. 

Applying the transformation of Equation (12.5), we get the matrix 
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42 %123 %4134 141 %1 212 *234 *241 %242 %2 S12 534 S42 
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Figure 12.16. A basis submatrix. 


Ei E, Ef-E\B,'D, E5-E5B;'D, E 
B, 0 0 0 0 |. (12.6) 
0 B, 0 0 0 


Here, B, consists of x,), x23, X34, and x,, whereas B, consists of x34, 
X41, X242, and x). These two rooted spanning trees are illustrated in Figure 
12.17. In addition D, and D> are represented by x, 4, and x22 and correspond to 
the cycles of Figure 12.17. 
Examining Figure 12.17, we see that 

0-(-]) | (1,2) 

0-(-I) | (2,3) 

E] -E,B;'D, =| 0-(-1) |(3,4) 

1-0 |(41) 

0-0 |(4,2) 


(b) 


Figure 12.17. Graphical illustration of the basis matrix: (a) Commodity 1 
sub—basis. (b) Commodity 2 sub—basis. 
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Figure 12.18. The transformed basis submatrix. 


1-0: 1:2) 
0-0 |(2,3) 
E5 —E5B3'D, =| 0-0 | (3,4) 
0-(-}) | 4D 


0-(4+1) |(4,2). 


Substituting this information into the transformed basis submatrix of matrix 
(12.6), we get the basis representation shown in Figure 12.18. 


12.6 SYNTHESIS OF MULTITERMINAL FLOW NETWORKS 


The network flow problems considered thus far have been network analysis 
problems, in which a certain network is given and we are then required to deter- 
mine some specified characteristic of this network. In contrast, the problem 
studied in this section is one of network synthesis or network design. Here, a 
certain required characteristic of the network is specified, and we have to 
synthesize or design a least cost network that satisfies the stated requirements. 
More specifically, the particular problem analyzed here is as follows, and 
arises in the design of communication networks. Suppose that we are to 
construct some undirected network on a specified set of nodes 1,..., m. This 
network can be designed by constructing links between node pairs i andj in 
some arc set .~ This set may or may not be totally dense, but we assume that if 
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all the potential arcs in .““ are constructed, then the resulting network will be 
connected. In addition to selecting the arcs from .*“ we have the freedom of 
designing the capacities of these arcs. If arc (i, /) is constructed with capacity 
Uys then a corresponding cost of c;; 2 0 per unit capacity will be incurred. 


The constructed network must independently sustain a certain maximal 
flow requirement 7; > 0 between each pair of nodes or terminals 7 and j (hence, 


the name of this problem). Note that the requirements ry are specified between 


each pair of nodes i and /, and not only for (i, /) € .~ The problem may be 
mathematically stated as follows, where f;; denotes the maximum flow between 


node pair i and j and is, of course, a function of the constructed network. 


Minimize{ > Cy if; 2 Ny for each? =1,...,.m—1,j=i+ 1,..., m, 
Gje7 


and uj 20 for (7, /)e.7}. 

Before proceeding to solve this problem, there are two manipulations that 
we need to perform. First, we do not need all the m(m — 1)/2 structural 
constraints in this problem. Intuitively, it should be evident that if the maximal 
flow requirements are satisfied for certain key node pairs, then the other flow 
requirements ought to be also automatically satisfied. In fact, these key node 
pairs turn out to be associated with a certain tree 7, known as the dominant 


requirement tree. Only the particular node pairs associated with the (m — 1) arcs 
in 7, need to have their flow requirement constraints explicitly represented. The 
second manipulation involves the representation of the constraints themselves. 
The quantities 7; are functions of the variables u,,. This translation must be 
made in order for us to be able to solve the problem mathematically. Note that 
this is readily done using the maximal flow—minimal cut theorem. Namely, for 
any node pair (i, j), fy; 2 7; if and only if the capacity of each cut separating 


nodes i and j exceeds rj (why?). (Note that since the network is undirected, we 


deal with the cut here, and there is no forward or reverse cut in this context.) Let 
k= 1,..., Ky index the possible cuts separating nodes i and j. For each such k € 


Kys 


each arc in .~~ with the component corresponding to arc (p, q) being | if this 
arc belongs to the cut and zero otherwise. Then, for any node pair (i, 7), we may 
replace fj; 2 7 by the constraints 


define a column vector aj, having |.°/ | components, one corresponding to 


(ajq)u 2h for k = 1...., Ky, 
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where u is the vector (ug, (i, j) € -7%). Before we consider the first 


manipulation, let us introduce an example problem. 
An Example for the Synthesis Problem 


Consider the potential network shown in Figure 12.19a, where m = 5, 


7 | =6 
with the permissible arcs as shown, and the corresponding costs cj per unit 


capacity as indicated against the arcs. Additionally, the requirements 7; are as 


follows: 
H2=3, n3=5, ng=3, As =2, m3 =6, 
4 = As 5 = 5, 4 = 8, 5 = 1, 145 = 12. 


Dominant Requirement Tree 7, 


To define the dominant requirement tree 7,, construct a complete graph G, on 
the node set 1,..., m, with the (undirected) arc between each node pair i and j 
having a weight equal to the requirement 7,;. Then, 7, is defined as the maximum 
spanning tree for G,., that is, it is a spanning tree for G, that has the maximum 


total weight. 
The process of constructing 7, involves a simple one—pass procedure and 


may be done as follows. Suppose that the “weights” hy are sorted from largest to 
smallest in a list L. (A complete sort is not necessary.) Select arcs for inclusion 
in 7, by proceeding in the order of list Z. Skip any arc that results in a cycle with 
respect to the arcs already selected. Stop when (m — 1) arcs have been selected. 
For the example problem, we would select and skip arcs as follows: select (4, 5), 
select (3, 4), select (2, 4), skip (2, 3) (why?), skip (2, 5), select (1, 3), and stop. 
The resulting tree 7. is shown in Figure 12.19b. 

The fact that this “greedy” procedure produces a maximal spanning tree 
T, may be verified as follows. Suppose that the tree 7” is some maximal 
spanning tree. We will show that the total weight of 7* and the constructed tree 
T, are the same. Toward this end, proceed down the list Z and find the first arc 


(p, q) that belongs to 7, but is not in 7". Add (p, q) to T” and examine the 
(unique) cycle formed by (p, g) and the chain C connecting p and q in T”. Note 
that 7 > Pyg for all (i, 7) € C since T* is maximal. If any (i, /) € C satisfies hj 
> Tpqg> then (i, /) € T,. If not, then because (i, /) appears before (p, q) in L and it 
was not selected for inclusion in 7, it must have formed a cycle with the arcs 


selected for 7,, before (i, j). But all such arcs are also in 7” by the definition of 


(p, g). Hence, we have a contradiction (why?). But all the arcs in C cannot also 
be in 7, since (p, g) € T,.. Hence, there exists an arc (s, ) € C, (s, 1) € T,, such 
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that r,, = ‘nq (why?). Now, add (p, q) to T* and remove (s, ft) from 7”. The 


resulting tree 7,.., is still maximal. However, the number of arcs in 7, that are 


new 


not in 7” 


new iS one less than the same number with respect to the previous tree 


T*. Continuing in this fashion, we will ultimately obtain a maximal spanning 


tree T” that coincides with 7,. 
The claim with respect to 7, is that if we satisfy fj; 2 7 for all (i,/) € 


T,, then we will automatically satisfy Sy 2 hy for ally 1 n= lg Sat 


1,...,m. To see this, suppose that for a constructed network, we have satisfied 
fij 2 %y for all Gj) € T,, and consider any (p, g) ¢ T,. Examine the chain C 


connecting nodes p and q in 7,. Then, we have r,, 


T, is maximal. But fj 2 7%j for all (i,j) € C since C is a chain in 7,. Hence 


< min {iyi Gs) € C} since 
> "pq 
< min { A ffm ee C}. In the constructed network, f,, equals the capacity 


u[X,X] of some cut [X,X] separating p and g. However, [X,X] is also a cut 


for some node pair s and ¢ with (s, #) € C (why?). (The node s may be p or the 
node ¢ may be q.) Consequently, using the maximal flow—minimal cut theorem, 
we have that 


fing =MX, X= fey 2 min (fi (i,j) € C} 2 rpg. 


This establishes the required result. 
Column-Generation Algorithm for the Synthesis Problem 


Using the foregoing two manipulations and denoting the vector of cost coeffi- 
cients (c;;, (i,j) € -Y) as ¢, we can formulate the multiterminal network design 


problem as follows: 


P: Minimize cu 


subject to (ayy) > ry for k =1,...,Kjy, for each (i, /) € 7, 


u>0. 


Denoting by w= (wx, k=1,..., Kj, (i,j) € T,) the dual variables associated 


with the structural constraints, we may write the dual to this problem as follows: 


Ky 
D: Maximize yee hy Wijk 
(i, eT, k=1 
Ki 
subject to DL (ayy Wye Se (12.7) 
G,j)eT, k=1 


w20. 
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Observe that similar to the Dantzig-Wolfe master problem of Chapter 7, this 
problem has several columns that are not all worthwhile enumerating explicitly. 
Hence, we adopt a column generation procedure in the context of a revised 
simplex algorithm. Because ¢ > 0, we commence with all slacks as basic in the 
starting basic feasible solution. Note that the corresponding complementary dual 
variables u associated with the constraints in Equation (12.7) are all zeros. At 
any stage, given a basic feasible solution to D with a simplex multiplier vector 
u, we need to price out the nonbasic variables. For the variable Wijk s the reduced 


cost “cy, — 244” is given by nj — (ajx )' UW. Because we are interested in finding 


the variable having the largest reduced cost, we examine the problem: 


Maximize { ry — (ijk ya} = ry ~ minimum (aj, a (12.8) 


FL hy 


for each (i, /) € 7,. Observe that this problem essentially seeks the minimum 
capacity cut separating i and 7 with respect to the capacity vector u (why?). 
Hence, this is equivalent to solving the maximal flow problem between nodes i 
and j on the current network using capacities u and determining the corre- 
sponding quantity fj. If fj 2 7,, that is, if the associated minimal cut gives a 
nonpositive value in Equation (12.8) for all (i, 7) € 7, then we are optimal. 
Otherwise, we may select the variable w,, that yields the most positive reduced 


cost in Equation (12.8) to enter the basis, pivot, and repeat. 


Summary of the Column—Generation Algorithm 


INITIALIZATION 


Construct a revised—simplex tableau for Problem D associated with the starting 
basic feasible solution having all the slack variables s, say, as basic. Hence, 


B"! =I is the basis inverse of size Fea |x| .-/ 


, the right-hand-side is ec, and 


the dual variable vector U is zero. 

MAIN STEP 

Ifany wu; <0, then the corresponding slack s,; is a candidate to enter the basis. 
Generate its updated simplex column and pivot it into the basis. Otherwise, if 7 
> 0 for all (7, /), examine the (constructed) network with capacity vector W and 
find the maximal flow j,; between each pair of nodes (i, /) € T,. If fy 2 nj for 
all (i, /) € T,, then stop; U is an optimal set of capacities. Otherwise, select the 
most violated constraint, say for (p, g) € 7,, and using the associated minimal 


cut k, generate the column a,,,, for the variable w,,,. From Equation (12.8), the 
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value “‘z < 0. Update the column a,,, (by premultiply- 


” —3 me 
pak ~€pgk” = Sq — "pq pqk 


ing with B! ), and perform the usual simplex pivot. Repeat the main step. 


Note that since this is a simplex based procedure, it is finitely convergent 
provided that some appropriate cycling prevention rule such as the lexicographic 
method is employed (see Chapter 5). 


Illustrative Example 


For the example of Figure 12.19, observe that the Problem D in Equation (12.7) 
is of the following form: 


Maximize 


X51 34 +27 Woak +2 81344 +21 2W45 
k k 


subject to 
C12 2 
C13 3 
ce 2 
© (134) 13k +2 (Arak Wok +2 (sae )34e +2 ase 45k = sh ie 
C35 3 
C45 2 


w20. 
Iteration 1 
B!= I,u=0, B~!c=c. The maximal flows ty = 0 for all (i, 7) € T,, and so the 
constraint f45; > 12 is most violated. Arbitrarily select the minimal cut LX, X] 
with X= {4}, X = {1, 2, 3, 5}. The corresponding column a4s5;, =a4s1 is (0, 0, 
0, 1, 0, 1)’, where the ones are in the positions corresponding to arcs (2, 4) and 


(4, 5). The pivot results in s4s leaving the basis. The updated revised simplex 
tableau is as follows: 


NWANWNHOL 
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Figure 12.19. Potential network and 7, for the example problem: 
(a) Network. (b) Tree 7,. 


512 
513 
523 
W24 
535 
W451 


Iteration 2 
Now, ws = 12, and #;; = 0, otherwise. We find that /45 = 12 and f;; = 0, other- 


wise, for (i, /) € T.. Hence, f4 = 0 <8 is most violated. A minimal cut [X,X] 
has X = {2, 3} and X = {1, 4, 5}, for example. The corresponding column for 
W341 iS 434, = (1, 1, 0, 1, 1, 0)’ and its updated version is shown with the fore- 
going tableau. The simplex pivot results in sy4 leaving the basis (using the 


lexicographic rule to break ties). This results in the following revised simplex 
tableau: 


512 
313 
523 
W34) 
535 
W451 


Note that currently, #24 = 8, uy5 = 4, and uy = 0, otherwise. This gives 


fia =9, fog =8 Ag =0, and fy5 = 4 as the maximal flow values. Hence, the 
constraints 34 > 8 and 45 = 12 are both most violated. We can now continue 
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by generating the appropriate column of w34) or w4s>. (Note that a sequential 
pivot of more than one improving column per iteration is also permissible.) We 
ask the reader to complete the solution of this example in Exercise 12.63. 

An Efficient Algorithm for a Special Case 


We conclude this section by presenting a particularly elegant algorithm for the 
special case of the foregoing synthesis problem in which all possible links are 
permissible, that is, .°/= {(i,/), 7=1,....m—1,j7=i+ 1,...,.m}, and also, cy = 1 


for all (i, j) € .*%“ Hence, we are supposed to design a network having the 
minimal total capacity that will sustain the flow requirements. The procedure for 
this case is presented below, using the example of Figure 12.19 as an 
illustration. 


STEP 1 


First, construct the dominant requirement tree T,. This is illustrated in Figure 
12.19b. 


STEP 2 


Decompose 7, into a “sum” of uniform requirement trees. Namely, construct a 
tree 7,; that is identical to 7, but with all arcs having the same weight as the 
smallest 7; value for (i, j) € T,. Let this weight be 5. Subtract 5 from all the 


ry-values for (i, /) € TJ, and drop the arcs in J. whose weight has reduced to 


zero. This results in a forest of two or more trees. Repeat with each proper tree 
in the resulting forest, creating the uniform requirement trees 7,),7,,....T,x in 


the process. 
For the tree 7, of Figure 12.19b, the uniform requirement trees are 


depicted in Figure 12.20a, where the (uniform) weights in each tree are shown 
against the arcs. Here, we obtain K = 4 uniform requirement trees. 


STEP 3 


For each uniform requirement tree 7,;, 7 = 1,..., K, design a graph G; that is an 


arbitrarily ordered undirected cycle on the nodes in 7,;, with each of the arcs in 


the cycle having a capacity u(G;) equal to acne the uniform requirement 
weight of 7,;. Then, an optimal network G* is simply a superposition (sum) of 
the graphs G;, i= 1,..., K. 

For the example, Figure 12.20b shows the cycle graphs G,,G,,G3, and 
G4. Figure 12.20c gives their sum G". 

In order to show that G* is indeed optimal, define R; = max {ny J# i 


for i = 1,..., m. Note that by the construction of T,., we have (why?) 
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Figure 12.20. Synthesis example for a special case: (a) Decomposition of 7, 
into uniform requirement trees. (b) Cycle graphs G; corresponding to T7,,;, i 


= 1,..., K = 4. (c) Optimal graph G’. 


R, = maximum {7,7 # i} = maximum {nj}. (12.9) 
LG /)ET, 
Hence, for our example, from Figure 12.19b, R; =5, Ry =7, R3 = max {5, 8} = 
8, Rq = max {7, 8, 12} = 12, and Rs = 12. 
Consider any graph G having arc capacities u;; that is feasible to the syn- 


thesis problem. Clearly, by feasibility, the sum of the capacities of the arcs 
incident at each node / must be at least R;. Hence, the total cost for G satisfies 


(12.10) 


Observe that the constructed graph G” is feasible, since by the construc- 
tion of the cycle graphs, we have Sy 2 ty for all (i, 7) € T,, (actually these 


hold as equalities), and so, Si 2h for all i=1,...,.m—1,j7=i+ 1,..., m (why?). 


Moreover, the sum of the capacities of the arcs incident at any node p in G* 
equals the sum of the uniform flows on the trees 7,; that contain node p. By the 
construction of the uniform requirement trees, the latter quantity equals the 
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maximum value of rp; over all j such that (yp, j) € T,. From Equation (12.9), 
this is precisely R,,. Hence, following Equation (12.10), the total cost for G* equals 


1/2 Di Ry which, from Equation (12.10), establishes the optimality of G*. 


Note that the flexibility in the algorithm admits several alternative 
optimal networks. Hence, it is pertinent to consider optimizing a secondary 
objective over all alternative optimal solutions to the present problem. Exercise 
12.65 presents a procedure for determining an optimal (minimal total capacity) 
network that is flow dominant, that is, it admits a maximal flow between any 
node pair that is at least as great as the value obtainable via any other optimal 
network. 


EXERCISES 


[12.1] Find the maxima! flow from node | to node 8 in the following network. 
Identify the associated minimal cut. 


[12.2] Consider the production process shown below that indicates the various 
paths that a product can take on its way to assembly through a plant. The 
number in each box represents the upper limit on items per hour that can be 
processed at the station. 


a. What is the maximal number of parts per hour that the plant can 
handle? 
b. Which operations should you try to improve? 


[12.3] Find the maximal flow from node | to node 7 in the following network. 
Identify the associated minimal cut. 
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[12.4] Discuss the economic meaning of the dual variables in the maximal flow 
problem. Interpret both the w;~— and the A, —variables. 


[12.5] Two paths are said to be arc disjoint if they contain no common arcs. 
Prove that the maximal number of arc disjoint paths from node 1 to node m ina 
network is equal to the minimal number of arcs that must be deleted in order to 
separate node | from node m. 


[12.6] In a command and control communications network a commander is 
located at one node and his subordinate at another node. Associated with each 
link in the network is an effort w,; required to eliminate that link from the network. 


a. Present a mathematical model that could be used to find the minimal 
effort to block all communications from the commander to his sub- 
ordinate. 

b. Indicate how the problem can be solved by a special algorithm. 


[12.7] Consider the maximal flow problem in which the variables are restricted 
as ~ 00 < Ly S xy< uy <. Write the dual to this problem. Based on this 


answer, define the capacity of a cut so that the maximal flow—minimal cut 
theorem holds in this case, assuming feasibility. (Hint: See Equation (12.2).) 


[12.8] Show that any basic feasible solution to the dual of the maximal flow 
problem (with node 1 taken as the root node) corresponds to a certain cut with 
the dual objective value equal to the capacity of the cut. Use this along with 
Lemma 12.1 to establish the maximal flow—minimal cut theorem. 

[12.9] Show that we have a basic feasible solution to the maximal flow problem 


if there exist no cycles in the set E = {(i, j): 0 < x; < uj}. In this case, show 


how the basic variables and the simplex tableau can be obtained at any iteration 
of the maximal flow algorithm. Furthermore, show that the following procedure 
will maintain basic feasible solutions in the maximal flow algorithm. 


Step 1. At each iteration, begin with E as defined earlier. 

Step 2. Try to find a path from node | to node m in G' associated only 
with arcs in E. 

Step 3. If no path is available from Step 2, then add one arc in G’ not 
associated with arcs in E to the set £ if it permits the labeling of 
a new node. With this new arc in £, return to Step 2. 
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[12.10] What simplifications would result if the network simplex method (of 
Chapter 9) is used to solve the maximal flow problem? Give all details. 


[12.11] Indicate how lower bounds on flows can be handled in the maximal 
flow algorithm. (Hint: Apply a Phase 1 procedure as follows. From G construct 
G" by: (1) All nodes in G are in G’. (2) In addition G' contains two new nodes 


m+ 1 and m + 2. (3) All arcs in G are in G’. (4) ty = 0, Uj; = uy — €y. (5) If 


£;; > 0, then place arc (i, m+ 2) in G’ with uj m4. =) and £5 m42 = 0; place 


arc (m + 1, j) in G’ with unis; = y and 41; = 0. (6) Solve for the 


maximal flow from node m+ 1 to node m + 2 in G’. (7) If at optimality all arcs 
out of node m + 1 are saturated (that is, Xi = Uj; ), then a feasible flow exists in 


G and xy = xij rey; otherwise, no feasible solution exists.) 


[12.12] Develop a dual simplex method for the maximal] flow problem. 


[12.13] Consider the following procedure for reducing the size of a network 
while finding the maximal flow from node | to node m. (Assume £;; = 0 and 


u,; > 0 for all (i, /).) 


Step 1. Remove all arcs entering node | (the source) and leaving node m 
(the sink). 

Step 2. Discard any node that has no arcs incident to it. 

Step 3. Discard any node, except node 1, that only has arcs leaving it. 
Also discard these arcs. 

Step 4. Discard any node, except node m, that only has arcs entering it. 
Also discard these arcs. 


Repeat Steps 2, 3, and 4 until no change results. If node | or m is discarded, 
stop; the maximum flow is zero. Otherwise, use the maximum flow algorithm. 


a. Show that the maximal flow in the resulting network is the same as 
in the original network. 

b. Is it true that there is a path from node 1 to every node in the 
resulting network (assuming that node 1 has not been discarded)? If 
not, then what additional operation is required to ensure this? 


[12.14] How can node capacities be handled in the maximal flow algorithm? 


[12.15] Suggest a scheme for selecting which node to label next in the “labeling 
algorithm” for the maximal flow problem so that at each iteration a maximal 
flow augmenting path is determined. Comment on the relative computational 
advantage of using such a scheme. 


[12.16] Exhibit in detail how the labeling algorithm for the maximal flow 
problem is equivalent to an application of the out-of-kilter algorithm, in which 
several primal phases are performed and the first dual phase adjusts the dual 
multipliers and terminates. Is this always the case when using the out—of—kilter 
algorithm for any general problem with only one initial out—of—kilter arc? 
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[12.17] Consider the problem of finding the minimum number of lines to cover 
all zeros in the assignment algorithm (refer to Section 10.7). Show that the 
maximal flow algorithm can be used to accomplish this task. (Hint: Given the 
reduced assignment matrix, construct a maximal flow network G as follows. Let 
nodes 1,..., n represent the » rows of the assignment matrix and nodes n + ],..., 
2n represent the » columns of the assignment matrix. If the ith entry in the 


reduced matrix is zero, draw an arc from node j to node n + j with u =o, 


int] 
Add two additional nodes 2n + 1 and 2n + 2. Add an arc (2n + 1, i) with 9,4); 
= 1 for i= 1,..., m and an arc (n + i, 2n + 2) with Uyyjan42 = 1 for i= 1,..., 0. 
Solve for the maximal flow from node 2 + 1 to node 2” + 2 in G. The value of 
the maximal flow is equal to the minimum number of lines to cover all zeros in 
the reduced matrix. To find which lines to use, consider the sets XY and X when 
the maximal flow algorithm stops. If (2n + 1, i) is in (¥,X), draw a line through 
row i. If (n + i, 2n + 2) is in (X,X) draw a line through column i. It still must be 
shown that this procedure works.) 


[12.18] Apply the procedure of the previous problem to find the minimum 
number of lines to cover all zeros in the following reduced assignment matrix. 


hwWN — 


[12.19] In this chapter we have provided the node—arc formulation for the 
maximal flow problem. Consider an arc—path formulation as follows. Let j = 
1,..., ¢ be an enumeration of all the paths from node | to node m in the network. 
Number the arcs from | to 7 and let 


__ 1 if arciisin path j 
Pi -)0 otherwise. 


The arc—path formulation for the maximal flow problem is given as follows: 


t 
Maximize >, x; 
j=l 
; t 
subjectto > pyx;Suj, i=1,....n 
j=l 


x; 20, J=1,...5¢, 


where x, represents the flow on path /. 


a. Give the complete arc—path formulation for the maximal flow 
problem of Figure 12.1. 
b. Solve the linear program of Part (a). 
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[12.20] Consider the arc—path formulation for the maximal flow problem as 
given in Exercise 12.19. Suppose that we do not enumerate any paths to begin 
with but, decide to apply the revised simplex method with all slack variables in 
the starting basic feasible solution. At any iteration of the revised simlex method 
let w be the dual vector. 


a. What is the simplex entry criterion for (i) a slack variable and (ii) a 
path variable? 

b. Show that there is an easy method to test the simplex entry criterion 
for path variables using the shortest path algorithm. 

c. If you always first enter slacks until no more slacks are eligible to 
enter, show that you may use the shortest path algorithm for non- 
negative costs to test the entry criterion for path variables. 

d. Describe the complete steps of the revised simplex method thus 
obtained. 

e. Apply the revised simplex method developed in this exercise to the 
maximal flow problem in Figure 12.1. 


[12.21] a. Give the dual of the arc—path formulation for the maximal flow 
problem as stated in Exercise 12.19. 
b. If we add the restriction that the dual variables must be 0 or 1, 
what interpretation can you give the dual problem? 
c. Interpret the dual solution obtained in Part (e) of Exercise 12.20. 


[12.22] Referring to Exercise 12.19, is the constraint matrix for the arc—path 
formulation of a maximal flow problem always unimodular? Prove or give a 
counterexample. 

[12.23] Modify the maximal flow algorithm to handle undirected arcs. 

[12.24] Find the shortest path from node 1 to every other node in the following 
network. Identify the shortest path tree obtained. 


[12.25] Find the shortest path from node 1 to all nodes for the following 
network using Dijkstra’s algorithm as well as using the PSP algorithm. Identify 
the shortest path tree obtained. 
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[12.26] Modify the shortest path algorithm for problems having mixed-sign 
costs to find the shortest path from every node to node m. Illustrate by finding a 
shortest path from every node to node 7 in the following network: 


[12.27] Find the shortest path from every node to node 6 in the following 
network. Identify the shortest path tree obtained. (Hint: Apply the shortest path 
algorithm with nonnegative costs in reverse.) 


[12.28] Find the shortest path from node | to all the nodes in the following 
network. Use the algorithms of both Sections 12.2 and 12.3. Identify the shortest 
path tree obtained. 
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[12.29] a. Apply the shortest path procedure to find the shortest path from 
node | to node 5 in the following network. 
b. What is the difficulty in Part (a)? 
c. Solve the problem by the network simplex method of Chapter 9. 
Compare with the result in Part (a). 


[12.30] Suppose that you are to find the shortest simple path from node | to 
node m in a network that may contain negative cost circuits. Furthermore, 
suppose that you have obtained an optimal solution to the minimum cost flow 
problem on this network after designating b, = 1, b,, =—1, and 5; = 0 otherwise, 


and setting bounds 0 < x; <1 on all the arcs (i, /) in the problem. How would 


you identify the simple path from | to m that has a flow of one unit in this 
solution? Why is this not necessarily the shortest simple path from 1 to m? 
Provide a mathematical formulation of the problem of determining a shortest 
simple path. 

[12.31] Consider the problem of determining the shortest path from node 1 to 
node m in a directed network having no negative cost circuit. How can you 
formulate and solve this as an assignment problem? (Hint. Think of a path in 
terms of assigning to each node the next node to visit. This ensures that any 
feasible assignment solution generates a path from | to m.) 


[12.32] a. Show how the shortest path algorithm can be used to find the 
longest path from node | to node m in a network. 
b. When finding the longest path, what assumption must be made to 
solve this problem in polynomial-time? 
c. Use the results of Part (a) to devise direct algorithms for the longest 
path problem under the assumption of Part (b). 


[12.33] In any project there are usually a set of activities that constitute the 
relationships specifying which activities must be completed before a given 
activity can start. Project management is concerned with the scheduling and 
control of activities in such a way that the project can be completed as soon as 
possible after its start. The critical path is a sequence of activities that limits the 
early completion time of the project. (It is generally activities on the critical path 
that project managers watch closely.) 

Consider the following activities with indicated completion times and 
precedence relationships: 


670 Chapter 12 


COMPLETION 
ACTIVITY TIME(DAYS) PREDECESSORS 

A 4 as 

B 7 _ 

Cc 2 D,E 

D 7 A,B 

E 9 B,F 

F 7 A 

G 4 B,C, F 


Find the critical path for this project and the associated project time. 


(Hint: Draw an arc with its own beginning and ending nodes for each activity 
with the arc cost equal to the completion time of the activity. If activity Q must 
precede activity R, then draw an arc from the ending node of activity Q to the 
beginning node of activity R with zero cost on the arc. Provide a starting node to 
precede all activities and a finishing node to succeed all activities. In the 
network thus obtained, the longest path (why not shortest path?) will be the 
critical path.) 


[12.34] Find both the shortest path and the longest path from node 1 to node 6 
in the following network: 


[12.35] For the label-correcting shortest path algorithm of Sections 12.2 and 
12.3, does every label—correction step correspond precisely with some primal 
simplex iteration? If not, what additional modification is required to the shortest 
path algorithm to make this correspondence precise? 


[12.36] Show that at optimality of the shortest path problem, w; — w,, repre- 


sents a lower bound on the cost of the shortest path from node i to node m for 
each /. 


[12.37] a. How can the shortest path algorithm be used to obtain a starting 
(not necessarily feasible) solution when the out—of—kilter algorithm 
is applied to a minimal—cost network flow problem with nonzero 
right—hand-side values? 

b. Is there any advantage to doing this? 


[12.38] Because Dijkstra’s shortest path algorithm for nonnegative costs is 
extremely efficient, it would be highly desirable to be able to convert a network 
having some negative cost arcs to an equivalent network with nonnegative costs. 
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Consider the following procedure for accomplishing this in a network G having 
m nodes. 


INITIALIZATION STEP 

Let ¢ = 1 (¢ is the iteration counter). 
MAIN STEP 

1. Let i= 1. 


2: Let c; = minimum c;. If ¢; < 0, replace cj by cj — ¢; for all j and 


replace c,; by cy + ¢; for all k. 


35 If i< m replace i by i+ 1 and return to Step 2. Otherwise, proceed to Step 
4. 
4, If all Ci > 0, stop; the equivalent network is obtained. Otherwise, if ¢ < 


m+ 1, replace t by t+ | and return to Step 1; if f= m+ 1, stop; there is a 
negative circuit in G. 
Upon completing this procedure, if all cj; > 0 we may apply Dijkstra’s shortest 


path algorithm to the equivalent network. (Note that although the proper path 
will be found, its length must be adjusted.) 


a. Show that the method works. 
b. Apply the method to the networks in Exercises 12.28 and 12.29. 
c. Show that if cj; <0 at iteration m + 1, then there is a negative cost circuit 


in G that includes node /. Is it possible to develop a labeling procedure to 
find such a negative cost circuit? 

d. What is the complexity of this procedure? Comment on its relative advan- 
tage. 


[12.39] In the shortest path algorithm for problems having mixed-sign costs, 
show that at optimality there always exists a path from node 1! to any node k 


along which w; =w; +c, provided that w, <0. Also, show that if wz, =, 


then no path exists from node | to node k. 

[12.40] Consider a network having upper bounds and costs (all lower bounds 
are zero). Suppose that we wish to find, among all maximal flows from node 1 
to node m, the maximum flow that minimizes the total cost. This is sometimes 
called the minimal—cost—-maximal flow problem. 


a. Give a linear programming formulation for the minimal—cost-maximal 
flow problem from node | to node m in a network. 
b. Show how the out—of-kilter method can be used to solve this problem. 


C: Apply Parts (a) and (b) to the following network to obtain the minimal— 
cost-maximal flow from node | to node 4: 
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[12.41] Consider the following procedure, from Busacker and Gowen, for finding 
the minimal—cost-maximal flow from node | to node m in a network having 
nonnegative costs and all /; = 0. 


INITIALIZATION STEP 


Let all xy = 0. 


SHORTEST PATH STEP 

From G construct G’ as follows. All nodes in G are in G’. If xy < Uy in G, 
place (i,j) in G' with Aj, = uy — xj and cj, = cy. If xy; > 0 in G, place (, i) in 
G' with Aj; = xj and cj; = —cj. Find a shortest path from node 1 to node m 


in G’. If no path exists, stop; an optimal solution is at hand. Otherwise, pass to 
the flow change step. 


FLOW CHANGE STEP 


Let A = minimum{ Aj: (i, /) is in the shortest path}. Adjust flows along the 


associated chain in G by A, increasing flows on arcs that have the same 
orientation as that of the path and decreasing flows on arcs that are against the 
orientation of the path. Pass to the shortest path step. 


a. Apply the algorithm to the example network of the previous problem. 

b. Prove that the algorithm converges to an optimal solution in a finite 
number of steps. It is necessary to show that (i) negative circuits never 
occur in G’ ; (ii) after a finite number of flow changes, no path from node 
1 to node m will exist, and (iii) on termination, an optimal solution is 
obtained. (Hint: Consider the flow in the network as a parameter and 
show that after each flow change we have the minimal—cost solution for 
that amount of flow.) 

Cc: What difficulties would occur when we admit negative costs? 


[12.42] Consider the following algorithm, from Klein, for finding the minimal— 
cost-maximal flow from node | to node m in a network having mixed-sign 
costs and all £;; = 0. 


INITIALIZATION STEP 


Find the maximal flow from node 1 to node m in G. 
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NEGATIVE CIRCUIT STEP 

From G construct G’ as follows. All nodes in G are in G’. If xy < uy in G, 
place (i, /) in G’ with A= a= ag aod Cy = ie If x > 0 in G, then place 
(j, i) in G’ with A ji = Xj and ci, = —¢j. Use the shortest path algorithm or the 
method of Exercise 12.38 to find a negative circuit in G’. If no negative circuit 


exists, stop; an optimal solution is at hand. Otherwise, pass to the flow change 
step. 


FLOW CHANGE STEP 
Let A= minimum {A,;: (i, /) is in the negative cost circuit}. Adjust flows along 


the associated cycle in G by A, increasing flows on arcs that have the same 
orientation as that of the circuit and decreasing flows on arcs that are against the 
orientation of the circuit. Pass to the negative circuit step. 


a. Apply the algorithm to the network of Exercise 12.40. 
b. Prove that the algorithm converges to an optimal solution in a finite 
number of steps. 


[12.43] Bob, Ed, and Stu are in a car pool. They each live at points |, 2, and 7, 
respectively, in the following network. They agree to meet at point 10 every 
morning at a certain time and proceed from there to their work in a single car. 
The numbers on the arcs represent the travel times in minutes. 


a. What is the fastest route for each man to the meeting point? 
b. How early (counting back from the meeting time) should each man 
leave? Would anyone be agreeable to give another a ride to point 10? 


[12.44] A single machine is needed to perform a specified function for the next 
four years, after which the function and machine will no longer be needed. The 
purchase price of a machine varies over the next four years according to the 
following table: 
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ONE YEAR TWO YEARS THREE YEARS 


YEAR NOW FROMNOW FROM NOW FROM NOW 
Purchase price $26,000 $35,000 $39,000 $48,000 


The salvage value of a machine depends only on its length of service and is 
given by the following table: 


LENGTH OF SERVICE 1YEAR 2YEARS 3 YEARS 4 YEARS 
salvage value $17,000 $7,000 $3,000 $1,000 


The annual operating cost varies with length of service, as follows: 


LENGTH OF SERVICE NEW 1 YEAR 2 YEARS 3 YEARS 
Annual operating cost $3,000 $5,000 $9,000 $18,000 


Construct a network in which a shortest path will yield an optimal policy of 
purchasing, operating, and salvaging machines over the next four years if 
management wishes to minimize the total cost. 


[12.45] a. Modify the shortest path algorithm for nonnegative costs to handle 
an undirected network. 
b. Apply the procedure of Part (a) to find the shortest path from node 1 
to node 5 in the following network: 


[12.46] Can the shortest path algorithm for problems having mixed-sign costs 
be modified to handle undirected arcs? 

[12.47] Apply the decomposition algorithm to the following three commodity 
minimal—cost flow problem: 


G11) (7,4,3,7,3,-2,2) Cae 


(6,2,5,1,0,-1,4) 


(Biq2PrqsP3q) Ug 4 pq #2 pq>¥3 pq >"pg 29°39 ) 
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[12.48] Consider a metropolitan city with the area divided into four zones and a 
highway network connecting the zones. Let the following matrix, called the 
origin—destination matrix, specify the travel requirements from each (row) zone 
to every other (column) zone: 


2 
10 


Ol} |Rl[Oo;h 


0 
2 
4 


1 
0 
2 
6 
2 
1 


Akh WN — 
WII] OO] WwW] apo 


Travel times and arc (upper) capacities are given as follows: 


ARC d,2) (2,4) (2,5) @,2) (4,3) 6,1) 6,4) 
Travel time (min) 20 40 20 22 12 17 13 
Capacit 45 39 37 47 36 25 12 


Find a minimal time traffic assignment in the network. 


AS 


42 > 

[12.49] Resolve the multicommodity minimal—cost flow problem of Figure 

12.10 with u534 = 4 instead of 3. Is it reasonable to expect this solution in 

practice? 

[12.50] a. Give a linear programming formulation for the two—-commodity 
maximal flow problem shown below (with no individual commod- 
ity upper bounds). 

b. Find the two commodity maximal flow in the network. 
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[12.51] How can lower bounds be handled in the multicommodity minimal-— 
cost flow problem? 


[12.52] Given an optimal solution obtained from the decomposition procedure 
for the minimal—cost multicommodity flow problem, indicate how the dual 
variables for the individual commodity constraints (Ax; =b; and x; <u,) can 
be recovered. Apply the procedure to the example problem in Section 12.4. 


[12.53] Discuss the economic meaning of the dual variables for the various 
constraints in the multicommodity minimal—cost flow problem (namely, ¥;x; < 


u, AX; =b;, and XxX; <u,). 


[12.54] Consider the multicommodity maximal flow problem without the indi- 
vidual capacity constraints x; <u, for i = 1,...,t. A disconnecting set is a gener- 


alization of a cut for the single-commodity flow problem. A multicommodity 
disconnecting set is a set of arcs that “disconnects” (cuts all paths between) the 
source and sink for every commodity. A multicommodity minimal disconnecting 
set is one for which the sum of (common) arc capacities is minimal. 


a. Give a mathematical formulation for the minimal disconnecting set 
problem. (Hint: Take the dual of the arc—path formulation for the 
maximal flow problem (see Exercise 12.19) and require the dual 
variables to be zero or |. Give an interpretation of this dual problem.) 

b. Show that the capacity of the multicommodity minimal disconnecting 
set is greater than or equal to the value of the multicommodity maximal 
flow. (Hint: Apply duality theorems to the formulation in Part (a).) 

c. Give a minimal disconnecting set for the network of Figure 12.9 and to 
the network of Exercise 12.50. 

d. Compare the capacity of the minimal disconnecting set and the value of 
the maximal flow for both problems of Part (c). 


[12.55] Show that a multicommodity minimal disconnecting set (see Exercise 
12.54) is the union of single-commodity forward cuts. Is a multicommodity 
minimal disconnecting set necessarily the union of single-commodity minimal 
forward cuts? 

[12.56] Discuss the difficulties, if any, in developing an algorithm for the 
multicommodity minimal—cost flow problem that begins with the minimal—cost 
flow for each commodity and proceeds to adjust these flows to satisfy the 
common upper bounds. 

[12.57] Give a node—arc formulation for the multicommodity maximal flow 
problem. Develop a decomposition procedure for this formulation and discuss 
the nature of the ith subproblem when x; <u; is present and when it is absent. 
[12.58] Develop an arc—path formulation for the multicommodity maximal flow 
problem without the presence of the constraints x; <u, for /= 1,..., 1 Develop a 
decomposition procedure for this formulation. (Hint: Consider the formulation 
given in Exercise 12.19.) 

[12.59] Modify the decomposition algorithm for the minimal—cost multicom- 
modity flow problem when the set X; = {x;:Ax; =b;,0<x; <u;} is not bounded. 
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(This is only possible when for some i at least one component of u, is 00; that 
is, there is no upper bound on some arc.) 

[12.60] How can undirected arcs be handled in the multicommodity maximal 
flow problem? Illustrate on the following three-commodity network: 


[12.61] In the multicommodity minimal—cost flow problem suppose that we 
have the capability of inverting the matrix [E” — E'B 'D, E]. Show how the origi- 


nal primal and dual variables can be obtained. Use this information to develop a 
simplex procedure for solving the multicommodity flow problem directly on the 
graph. Give all details and illustrate by the problem of Figure 12.10. 

[12.62] Consider the problem of synthesizing a network having the following 
structure, where the costs per unit capacity are given on the arcs: 


a. Determine a set of optimal capacities for these arcs in order to be able to 
sustain the following flow requirements: 42 = 6, 43 =3, Ag = 2, m3 = 4, 


oy am 2; and Ra = 7. 
b. Repeat, assuming that arc (1, 4) can also be constructed and that cy = 1 for 


all (i, j). 


[12.63] Determine an optimal solution to the network synthesis example in 
Section 12.6. 


[12.64] Let F be a symmetric nonnegative matrix of size m x m with elements 
fy, where f,; = 0 for i= 1,...,.m. Then F is said to be max flow realizable if 


there exists some undirected graph on m nodes for which f, is the maximal 
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flow between nodes 7 and j. Show that F’ is max flow realizable if and only if fj 
> minimum {f;;, Sig} for all i, 7, &. Furthermore, show that if F is realizable, 


then it is realizable by a tree graph. What does this say about the maximum 
number of distinct maximal flow values between pairs of nodes in any 
undirected graph? (Hint: Section 12.6 proves that this condition is necessary. To 
prove that it is sufficient, construct a maximal spanning tree for a complete 
graph with each arc (i, /) having a weight fj.) 


[12.65] Consider the problem of synthesizing a complete undirected graph hav- 
ing a minimum total capacity that satisfies given flow requirements 7; between 


node pairs i and . Let G* be an optimal graph, and let fi be the maximal flow 


between node pairs i and j in G". Now, let R; = maximum {7;,, 7 # i} be as 


"ip 


defined in Equation (12.9), and compute i = minimum {R,,R,;} for all node 


pairs i and j. Denote by G™* an optimal graph obtained by assuming that the 


. x. . 
requirements are 7;; in lieu of 


ij hj for all (i, /), and let fi be the maximal flow 


between node pairs i andj in G™*. Show that G™ is also optimal for the problem 


with requirements 7,, and that fj 2 Si for all node pairs 7 and j. Interpret the 


significance of G™*. Construct G™ for the example of Section 12.6 and for the 


example of Exercise 12.62. (Hint: First show that (a) hij < minimum {R;,R ni 


for all (i, /), and (b) R; = maximum {rij , J #i} = R, for all i. Use these proper- 


ties to establish the required results.) 


NOTES AND REFERENCES 


1. Ford and Fulkerson [1956] were the first to develop the maximal flow 
algorithm for networks. Dantzig and Fulkerson [1956] provided a proof 
of the maximal flow—minimal cut theorem. Glover et al. [1984a, b] stud- 
ied specialized primal simplex algorithms for the maximal flow problem, 
and have shown them to be significantly (1.5—5 times) faster than the 
presented labeling techniques, while requiring one-third the amount of 
storage. They have also shown that this specialized scheme is roughly 1.8 
times faster than a version of a general primal simplex code RNET, even 
when it is tuned for maximal flow problems. For determining the maxi- 
mal flows between all pairs of nodes in an undirected graph via an 
equivalent tree network, see Gomory and Hu [1961]. For a general 
detailed discussion on scaling algorithms see Ahuja et al. [1993]. Also, 


the strongly polynomial-time method of complexity O(n?m) based on 
shortest path flow augmentations is due to Edmonds and Karp [1972]. By 
far, the most effective algorithms for solving maximal flow problems are 
the preflow—push methods (see Shiloach and Vishkin [1982] and Ahuja et 
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al. [1993]. Wherease preflow algorithms permit flow imbalances only of 
the type where the excess given by the inflow minus the outflow is 
nonnegative, a more recent algorithm due to Hochbaum [2008] is based 
on pseudoflows where nodes can have excess or deficit (negative excess) 
flows. This algorithm, which is of complexity O(mnlog(m)), first 
determines a minimal cut through an equivalent blocking—cut problem, 
and then recovers the maximal flow (with additional complexity O(n 
log(m))). 

2. The first algorithms for shortest path problems were developed by 
Bellman [1958], Dijkstra [1959], Dantzig [1960], Whiting and Hiller 
[1960], and Floyd [1962]. A good comparison of these procedures 
appears in Dreyfus [1969]. More efficient methods have been developed 
by Glover et al. [1985a, b]. Computer implementations of these 
algorithms are described by Klingman and Schneider [1986]. A 
comprehensive computational study appears in Glover et al. [1984a, b] 
and Hung and Divoky [1988], where it is shown that threshold based 
partitioned shortest path algorithms dominate other procedures. (Also, see 
Bertsekas [1993].) A good exposition on the complexity behavior of 
some shortest path implementations is given by Shier and Witzgall 
[1981]. Sherali [1991] presents relationships between the partitioned 
shortest path algorithm and a dynamic programming routine. Also, see 
Fredman and Tarjan [1984] for improved implementations using 
Fibonacci heaps. (Additional discussion on heap implementations 
appears in Ahuja et al. [1993].) For the use of decomposition techniques 
in solving large-scale shortest path problems on special networks, see Hu 
[1968], Shier [1973], and Jarvis and Tufekci [1981]. For a more detailed 
exposition on the relationship between shortest path algorithms and the 
simplex method, see Akgul [1986b], Dial et al. [1979], and Zadeh [1979]. 
The procedure of Exercise 12.38 was developed by Bazaraa and Langley 
[1974], and is based on ideas presented by Nemhauser [1972]. For 
generalizations of label setting and label correcting algorithms for 
bicriteria shortest path problems, see the survey paper by Skriver [2000]; 
for characterizing Pareto optimal (non-dominated) solutions to 
bicriteria/multi—criteria shortest path problems see Muller-Hannemann 
and Weihe [2006]; for time—dependent shortest path problems see Cook 
and Halsey [1966], Dreyfus [1969], Halpern [1977], Orda and Rom 
[1990], and Sherali et al. [1998]; for label—constrained and approach— 
dependent variants of this problem see Barrett et al. [2001] and Sherali et 
al. [2003, 2006]; for stochastic shortest path problems see Orda et al. 
[1993], Polychronopoulos and Tsitsiklis [1996], and Hutson and Shier 
[2009]; for complexity and algorithmic analyses of robust shortest paths 
under uncertainty, see Montemanni et al. [2004], Yu and Yang [1998], 
and Zielinski [2004]; and for dynamic and dynamic stochastic shortest 
path problems, see Psaraftis and Tsitsiklis [1993] and Cheung [1998a, b], 
respectively. Pallottino and Scutella [2003] discuss sensitivity analysis 
issues under changing arc costs for shortest path problems. Zhang and 
Yixun [2003] describe the reverse shortest path problem of finding a least 
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cost decrement in link costs in order to achieve a specified target 
objective value. Also, Sherali and Hill [2009] discuss a time-restricted 
reverse shortest path problem of determining a reverse shortest path 
under time-restricted arc availabilities and given a specified lower bound 
on the start-time. (Such problems arise in air traffic management.) 

Ford and Fulkerson [1958b] first proposed a column generation 
procedure for the multicommodity maximal flow problem. This was the 
forerunner to the Dantzig-Wolfe decomposition procedure for general 
linear programs. Hartmand and Lasdon [1972] proposed a procedure 
based on the simplex method for solving multicommodity flow problems. 
For the various exercises on multicommodity maximal flow problems, 
see Robacker [1956], Ford and Fulkerson [1958b], Bellmore et al. [1970], 
Grigoriadis and White [1972a, b], Hartman and Lasdon [1972], and 
Jarvis and Keith [1974]. A good discussion and survey on multicom- 
modity flow problems appears in Kennington and Helgason [1980], and 
Jones et al. [1993]. Lustig and Li [1992] provide excellent implementa- 
tion concepts. 

The network synthesis problem described in Section 12.6 is from Gomory 
and Hu [1962]. Also, see Ford and Fulkerson [1962]. 
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decomposition, 340, 642 
Dijkstra, 620 


dual simplex, 499 
Fourier—Motzkin, 144 
generalized network simplex, 494, 
512 
good, 449 
heap Dijkstra, 633 
interior point methods, 429 
Karmarkar, 402, 409 
Khachian, 401, 437, 450 
Klein, 672 
labeling, 482, 589, 613, 633 
Lagrangian relaxation, 371, 373 
maximal flow, 611, 678 
multicommodity flows, 639 
negative circuit, 632, 634 
network simplex, 453, 474 
network synthesis, 607, 654, 680 
out—of-kilter, 570, 573, 586, 605 
partitioned shortest path, 635 
path—following, 431 
predictor—corrector, 435 
primal—dual path following, 431 
primal—dual simplex, 285 
projective, 402, 450 
revised simplex, 202 
self—dual, 333 
shortest path, 546, 564, 620, 625, 
639, 679 
simplex, 91, 120, 466, 522 
threshold partitioned shortest path, 
637 
transportation, 522 
Alternative optimal solutions, 20, 92, 
115 
Algebra of the simplex method, 108 
Algebraically redundant, 162 
Alternating path basis algorithm, 544, 
564 
Analysis: 
big—M method, 165, 172 
convex, 45 
parametric, 312 
postoptimality (sensitivity), 295 
two-phase method, 154 
Angular structure, 361, 362 
Antecedent nodes, 483 
Anticycling rules, 186 
Applications of linear programming: 
aircraft assignment problem, 331 
air quality problem, 37 
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assignment problem, 535 

capital budgeting, 11 

copper market, 510 

coal blending and distribution, 15, 
36 

cutting stock, 10, 191, 384 

discrete control problem, 386 

distribution problem, 37 

equilibrium problems, 38, 324 

facility location problem, 30 

feed—mix, 8, 29 

game theory problems, 41, 324 

housing renewal planning problem, 
34 

investment problem, 29, 31, 34 

maximal flow problem, 607 

menu planning problem, 30 

minimal—cost maximal flow 
problem, 672 

minimal-cost network flow 
problem, 453 

multicommodity flow problem, 639 

personnel training problem, 30 

production scheduling, 9, 31, 33 

production inventory, 37 

production—transportation— 
inventory problem, 505 

project management problem, 669 

project selection problem, 34 

rocket launching problem, 35 

scheduling problems, 9, 31, 32, 33, 
36 


tanker scheduling, 13 
tourism problem, 332 
traffic assignment problem, 37, 675 
transportation problem, 11, 32, 513 
two-stage stochastic problem with 
recourse, 39 
warehouse location problem, 388 
Approach—dependent shortest path 
problem, 639, 679 
Arborescence, 456, 622, 631 
Arc: 
capacities, 608 
definition, 453 
in-kilter/out—of-kilter, 561 
label eligible, 569 
one—arcs, 545 
zero—arcs, 539 
Arc disjoint paths, 664 
Arc-path formulation, 608, 666 
Artificial intelligence modeling 
approach, 42 
Artificial technique: 


Index 


single constraint, 293 

single variable, 173 
Artificial variable, 153, 198, 522 
Assignment problem: 

alternating basis, 544, 564 

covering, 540, 560, 666 

definition, 535 

dual problem, 537 

finite convergence, 543, 550 

independent cells, 540, 560, 666 

(Kuhn's) Hungarian algorithm 

modifying dual variables, 540 

partial solution, 540 

polytope, 537, 561, 564 

problem, 535 

reduced matrix, 538 

successive shortest path, 546 
Assumptions in linear programming, 3 
Augmented matrix, 56, 61 
Augmented threaded index, See also 

Data structures, 482 
Average—case complexity, 449 


Back substitution, 56 
Balanced transportation problem, 514 
Barrier function, 444 
Barrier problem, 431 
Basic feasible partition, 222 
Basic feasible solution: 
bounded variables, 222 
definition, 62, 94, 95 
degenerate, 95, 98, 222 
existence, 101 
improving, 111, 223 
initial, 129, 151, 293, 475, 489, 522 
nondegenerate, 95, 222 
number of, 97, 224 
optimal, 21, 104, 225 
relationship to extreme points, 99 
Basic dual solution, 282 
Basic solution, 62, 95 
Basic variables, 95, 222 
Basis: 
adjacent, 106 
alternating path, 544, 564 
assignment problem, 561 
compact, 221 
complementary dual, 282 
crash, 173 
definition, 49, 61, 95 
entry criterion, 106, 111, 121, 178, 
180, 225, 493 
exit criterion, 112, 121, 178, 180, 
479, 527 
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generalized network, 494, 512 
Karmarkar's algorithm, 424 
matrix, 61, 95 
maximal flow problem, 613 
multicommodity flow problem, 649 
network flow problem, 461, 504 
number of, 97, 224 
optimal, 21, 104, 225 
relationship to trees in networks, 
461, 463, 518 
transportation, 518 
triangularity of in networks, 463, 
518 
working, 221 
Basis equivalent chain, 465, 469 
Benders’ partitioning procedure, 371, 
372, 388 
Bidirectional search, 452 
Big—M method: 
analysis, 167 
comparison with two-phase 
method, 168 
description, 165, 172 
Binary encoding, 394 
Binding constraints, active, tight, 71 
Bipartite graph, 514, 536 
Bland’s cycling prevention rule, 180, 
184, 198, 230 
Block diagonal constraint matrix, 361, 
362 
Blocking hyperplane, constraint, 106 
Blocking variable, 106, 112 
Block pivoting, 134, 599, 613 
Bounded set, 20, 75 
Bounded variable, 220, 478 
Bounded variables affine scaling 
algorithms, 446 
Boxstep method, 374, 390, 392 
Branch—and-price, 392 
Breakthrough, 576, 586, 594 
BTRAN, 209 
Bumps, 219 
Busacker-Gowen algorithm, 672 


Canonical form: 

of duality, 259 

of linear program, 5, 71, 104 

of simplex tableau, 126 
Capacitated network, 454 
Capacitated transportation problem, 

559 
Capacity: 

arc, 478, 608 

cut, 609 
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disconnecting set, 676 
forward cut, 609 
Capital budgeting problem, 11 
Caratheodory theorem, 77 
Central path, 432 
Chain: 
basis equivalent, 465, 469 
in graph, 456 
in transportation tableau, 517 
Change: 
in basis, 50, 109 
of constraint coefficients, 298 
of cost vector, 296 
of right-hand-side, 297 
Chord, 70 
Circuit, 456, 634 
Circulatory network flow problem, 568 
Circulation, 568 
Class P of problems, 396 
Closed chain, 456 
Closed interval, 29 
Closed path, 456 
Closed system, 439 
Cofactor, 59 
Column generation scheme, 339, 343, 
372, 657, 658, 680 
Column generation, stabilized, 344, 
374, 391, 392 
Column pivoting, 250 
Column rank, 61 
Column simplex method, 250 
Column vector, 45 
Combination: 
convex, 64 
linear, 48 
Communication network, 654 
Compact basis, 221 
Compact form, 214 
Comparison: 
of simplex and revised simplex, 
205 
Complementarity theorem, 327 
Complementary 
basic dual solution, 282 
pair of variables, 269 
slackness conditions, 239, 268, 538, 
569 
slackness theorems, 268, 336 
Complete graph, 456, 514 
Complexity: 
average—case, 449 
computational, definition, 81, 394 
genuinely (strongly) polynomial, 
396, 450, 617 
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of Karmarkar's algorithm, 418 
of Khachian's algorithm, 437 
order, 394 
polynomial, 394 
of PSP algorithm, 636 
of shortest path problem, 617, 622, 
633 
of simplex method, 397 
strongly (genuinely) polynomial, 
396, 450, 617 
Complicating constraints, 339 
Complicating variables, 371 
Computational complexity, 394, 397, 
418 
Component of graph, 456 
Concave function, 70 
Cone: 
convex, 68, 237 
generated by vectors, 69 
polyhedral, 71 
recession, 74 
Connected graph: 
strongly, 456 
weakly, 456 
Conserving flow, 567, 595 
Constraint: 
active, binding, tight, 71 
artificial, 293 
complicating, 339 
coupling, 386 
definition, 2 
functional, 2 
matrix, 2 
nonnegativity, 2 
Construction of basic feasible 
solutions, 94 
Consumption column, 301 
Control variable, 9 
Convergence: 
assignment algorithms, 543, 552 
bounded variable simplex method, 
229 
dual simplex method, 285 
interior point methods, 429 
Karmarkar's algorithm, 418 
Khachian's algorithm, 439 
maximal flow algorithm, 611 
out—of—kilter algorithm, 585, 591 
primal—dual method, 292 
shortest path algorithms, 622, 630, 
633 
simplex method, 122, 181 
Convex: 
analysis, 45 
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arc costs, 605 
combination, 64 
cone, 68 
function, 64, 70 
set, 64 
Convexity constraint, 362 
Coordinate ascent, 605 
Coordinate vector, 46 
Corner point, 72 
Corners of cycle in transportation 
tableau, 517, 520 
Corrector step, 435 
Cost coefficient, | 
Court scheduling problem, 36 
Covering in assignment problem, 540, 
560, 666 
CPLEX, 511, 605 
Cramer's Rule, 60 
Criss—cross algorithm, 333 
Criterion function, | 
Critical path problem, 600, 669 
Cut, 305, 372, 609 
Cut-set, 609 
Cutting plane algorithms, 305, 372 
Cutting stock problem, 10, 191, 384 
Cycle: 
in graph, 456, 575, 634 
in transportation tableau, 517, 520 
Cycle method for computing z j — cj, 
525, 564 
Cycling: 
example, 175 
geometry, 177, 199, 337 
in networks, 482, 509 
phenomenon, 106, 229 
practical prevention rule, 180, 198 
prevention rules, 178 
validation of prevention rules, 182 


Dantzig's Rule, 121 
Dantzig—Wolfe decomposition method, 
340, 642, 680 
Data structures, for network flows: 
antecedents, 483 
augmented threaded index, 482 
down, 482 
DUAL vector, 485 
final of node, 484 
grafting, 486 
FLOW vector, 485 
immediate successor, 483 
last successor, 484 
list structures, 482 
level, 482 


Index 


lower tree, 488 
next, 483 
ORIENT vector, 484 
postorder traversal, 483 
predecessor, 482 
preorder distances, 483 
preorder traversal, 483 
reverse thread, 483 
subtree rooted at node, 482 
successors, 482 
thread, 482 
upper tree, 488 
Decision problem, 396, 436 
Decision variables, 2 
Decomposition algorithm: 
algorithm/principle, 339 
block diagonal structure, 361 
economic interpretation, 369 
getting started, 353 
lower bound on objective function, 
345, 364 
multicommodity network flow 
problem, 641 
nested, 384, 385 
network synthesis problem, 607, 
654, 680 
unbounded subproblem region, 354 
Defining: 
hyperplane, 72 
variable, 104 
Degeneracy: 
in assignment problem, 537 
in basic feasible solutions, 72, 95, 
98, 221, 222, 229 
in networks, 488 
order, 72 
relationship 
to cycling, 175 
to shadow prices, 273 
in transportation problems, 523, 
528, 531 
Degenerate iteration/pivot, 106, 229, 
230 
Degree of node: 
definition, 457 
in—degree, 457 
out—degree, 457 
Degrees of freedom, 104 
Density, 206 
Dependent: 
constraints, 61 
variables, 95, 221 
vectors, 49 
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Destination in transportation problems, 
513 
Determinant of matrix, 59 
Deterministic assumption, 3 
Digraph, 453, 455 
Dijkstra's algorithm, 620 
Dimension: 
of basis, 50 
of Euclidean space, 48 
full, 73 
of matrix, 51 
of set (region), 73 
of vector, 45 
Directed: 
arc, 453 
cycle, 456 
network, 453 
Direction: 
associated with unbounded 
solution, 118 
of convex set, 66 
distinct, 68 
extreme, 68, 71 
of polyhedral set, 66 
of ray, 66, 118 
recession, 66, 71 
Disconnecting an arc, 457 
Disconnecting set, 676 
Discrete control problem, 386 
Discrete optimization, 4 
Distribution problem, 38 
Divisibility assumption, 3 
Dominant requirement tree, 655, 656 
Dot product, 47 
Dual: 
affine scaling, 431 
angular structure, 384 
canonical form, 259 
complementary basis, 282 
feasibility and primal optimality, 
239, 243 
formulation, 259 
mixed forms, 262 
of assignment problem, 537 
of circulatory network—flow 
problems, 569 
of dual, 261 
of maximal flow problem, 610 
of out—of—kilter formulation, 569 
phase, 577, 587 
problem, 239 
relationship with primal problem, 
264 
simplex method, 279 
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standard form, 260 
variables, 239, 260, 269 
Duality: 
Fundamental theorem, 267 
and Karush—Kuhn-Tucker 
conditions, 265 
and Lagrangian multipliers, 239, 
243, 371 
economic interpretation, 270 
gap, 324, 592 
involutory property, 262 
origin, 265 
strong, 266 
supervisor's principle, 268 
theorems, 267, 268 
weak, 264 
Dual feasibility, 239, 278 
Dual simplex method: 
bounded variables, 333 
development, 277, 499 
finite convergence, 285 
getting started, 293 
summary, 279, 281 
Dual variable method, 525 
Dynamic shortest path problems, 679 
Dynamic stochastic shortest path 
problems, 639, 679 
Dynamic trees, 512 


Economic interpretation: 
of decomposition, 369 
of duality, 270 
Edge 
of polyhedron, 73 
of graph, 454 
Efficient solutions, 8 
Elementary matrix, 207 
Elementary matrix Operations, 54 
Elimination form of the inverse, 256 
Empty feasible region, 22 
Encoding: 
binary, 394 
stroke, 396, 617 
End node of a tree, 457, 518 
Entry criterion, 106, 111, 121, 178, 
180, 225, 493 
Epigraph, 87 
Equal flow constraint, 511 
Equality constrained polyhedron, 82 
Equilibrium, 39 
Equivalent weights, 199 
Eta vector, 208 
Euclidean norm, 47 
Euclidean space, 48 
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Excess flow, 679 
Excess function, 594 
Exit criterion, 112, 121, 178, 180, 479, 
527 
Extreme direction, 68, 71, 74 
Extreme point: 
adjacent, 73, 254 
definition, 64, 71, 72 
optimality at, 91, 114 
relationship to basic feasible 
solutions, 99 
representation theorem, 75, 76 
Extract a node/arc, 628, 633, 635 
Extreme ray, 68, 71, 74 


Face: 
improper, 73 
proper, 71, 73 


Facility location, 30 
Factorization: 
interior point methods, 450 
LU, 212 
QR, 443 
Fair market price, 271, 273 
Farkas' Lemma, 234, 235 
Feasible: 
flow, 567 
region/space, 2 
solution, 2 
system, 23 
Feed mix problem, 8, 29 
Fibonacci heaps, 679 
Final node, 484 
Finite convergence, see Convergence 
Finite optimal solution, 91, 114 
First—in—first—out, 635, 638 
Flow: 
in arc, 454 
augmentation, 548, 568 
conservation equations, 454 
excess, 679 
maximal, 607 
minimal—cost, 454 
multicommodity, 639 
with gains, 510, 512 
Forest graph, 457 
Forward arcs of a cut, 609 
Forward cut, 609 
Forward-star, 455, 628 
Fourier—Motzkin elimination method, 
144 
Fractional part, 304 
From-—node, 455 
FTRAN, 209 
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Full dimensional, 73 

Full rank matrix, 61 

Functional constraint, 2 
Fundamental theorem of duality, 267 


Game theory, 42, 324 
Gaussian reduction, 56, 63, 214 
Gaussian reduction matrix, 214 
Gaussian triangularization, 212 
Gauss—Jordan reduction, 56, 62, 212 
Generalized linear programming 
problem, 385 
Generalized networks, 512, 494 
Generalized transportation problem, 
558 
Generalized upper bounding, 257 
General solution of linear equations, 62 
Genuinely (strongly) polynomial, 396, 
450, 617 
Geometric interpretation: 
Farkas' lemma, 236 
Karush—Kuhn—Tucker conditions, 
237 
Geometric redundancy, 71, 190 
Geometric solution of linear programs, 
18, 104 
Geometry of cycling, 177, 199, 337 
Gomory's dual fractional cut, 305 
Gradient, 29, 65 
Grafting, 486 
Graph: 
bipartite, 514, 536 
complete, 456, 514 
component, 456 
connected, strongly, weakly, 456 
definition, 455 
digraph, 455 
forest, 457 
mixed, 455 
proper, 455 
tree, 456 
undirected, 455 


Half line, 69 

Half-space, 65, 66 

Heap Dijkstra procedure, 633 

Heap implementation, 633, 637, 679 

Hidden networks, 512 

Hirsch conjecture, 449, 564 

Hitchcock (transportation) problem, 42, 
514 

Homogeneous system, 74, 267 

Housing renewal planning problem, 34 
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Hungarian (Kuhn's) method, 535, 541, 
564 
Hyperplane: 
definition, 65 
normal (gradient) to, 65 
Hypograph, 87 
Identity matrix, 52 
Immediate successor, 483 
Improving basic feasible solutions, 
111, 223 
Inputed values, 273 
Inactive constraint, 71 
Incident, 454 
Inconsistent system, 21, 22, 190 
Incremental cost, 271 
In—degree of node, 457 
Independent cells in assignment 
problem, 540, 560, 666 
Independent variables, 95, 221 
Independent vectors, 48 
Induced by node set, 456 
Infeasible, 21 
Initial basic feasible solution, 129, 151, 
293, 475, 489, 522 
In-kilter, 570 
Inner optimization problem, 371 
Inner product, 47 
Input length of problem, 394 
Input—output management, 259 
Insert operations, 633 
Instance of problem, 394 
Integer part of coefficient, 304 
Integer programming problem, 304, 
536 
Integer property: 
in assignment problems, 536 
in network flow problems, 463 
in transportation problems, 517 
Integer variable, 304 
Interior point methods, 429, 605 
Intermediate node, 454 
Interval: 
closed, 29 
open, 29 
of uncertainty, 397 
Inverse matrix: 
calculation, 56 
condition for existence, 56 
definition, 56 
from simplex tableau, 132 
product form, 140, 207 
Investment problem, 29, 31, 34 
Involutory property, 262 
Irreducible infeasible system, 43 
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Irrelevant constraint, 71 
Iteration, 106, 109, 473 


Karmarkar's (projective) algorithm: 
complexity analysis, 418 
convergence, 418 
description, 402 
determining an optimal basis, 424 
form of linear program, 414 
potential function, 420 
sliding objective method, 424 

Karush—Kuhn—-Tucker conditions: 
for equality constraints, 241 
geometric interpretation, 237 
for inequality constraints, 237 
optimality conditions, 265 
perturbed, 432 
proof, 237 
relationship to: 

duality, 265 
simplex method, 242 

Key for heaps, 633 

Khachian's algorithm, 401, 437, 450 

Kilter: 
number, 571 
state, 571, 572 

Kirchhoff equations, 454 

Klee—Minty polytope, 398, 399 

Klein's algorithm, 672 

Knapsack problem, 138, 246 

Kuhn's Hungarian algorithm, 537 


Label-correcting algorithm, 628 
Label eligible arc, 576 
Labeling algorithm: 
maximal flow, 613 
network simplex, 482 
out—of-kilter, 589 
shortest path, 633 
Label—constrained shortest path 
problem, 679 
Labeling method, 607 
Label-setting algorithm, 621, 624, 628 
Lagrangian dual problem, 335, 374 
Lagrangian multipliers, 239, 243, 371 
Lagrangian relaxation, 371, 373 
Lagrangian subproblem, 374 
Last successor, 484 
Leading node, 474 
Leaf node, 457 
Least—recently—considered rule, 181, 
491 
Least squares problem, 407 
Leaving variable, 106 
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Left-hand shadow price, 275 
Legitimate variables, 153 
Length of a stage, 181 
Leontief input-output model, 42 
Level index, 482 
Lexicographically nonnegative vector, 
182 
Lexicographic ordering, 198 
Lexicographically positive vector, 182 
Lexicographic cycling prevention rule, 
178, 285 
Linear: 
dependence, 49 
independence, 48 
subspace, 48 
Linear combination, 48 
Linear equations: 
basic solution, 62 
Gaussian reduction, 56, 63 
general solution, 62 
number of solutions, 62 
redundant, dependent, 61 
Linear fractional program, 404 
Linear inequalities, 2, 4 
Linear programming problem: 
assumptions in, 3 
canonical form, 5, 6 
examples, 7 
formulation, 7 
generalized, 385 
geometry, 18 
standard form, 5, 6 
Linear subspace, 48 
Linear transformation, 404, 652 
Line belonging to set, 90 
Line search problem, 604 
Line segment, 64 
Link (arc), 453 
List structures, see Data structures 
Longest path problem, 669 
Low-order polynomial bound, 395 
Lower bounds: 
on objective function, 345, 364 
on variables, 220, 478, 655 
Lower tree, 488 
Lower triangular matrix, 53 
LU decomposition/factorization, 212 


Machine location problem, 29 
Machine scheduling problem, 31, 33 
Manhattan distance, 30 
Manipulation of linear program, 4 
Marginal cost, 471 

Master array, 342 
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Master Problem, 339, 340, 355, 362, 
371 

Matching, 536 

Mathematical model, 7 


Matrix: 
addition, 52 
adjoint, 60 


augmented, 56 

definition, 51 

determinant, 59 

diagonal, 58 

elementary, 207 

Gaussian reduction, 56, 63, 214 

generator, 220 

identity, 52 

inverse, 56, 63 

multiplication, 46, 52 

multiplication by a scalar, 52 

nonsingular, 56 

operations, 52 

partitioned, 53 

permutation, 214 

pivot, 214 

positive definite, 438 

postmultiplying, 208 

premultiplying, 208 

rank, 61 

singular, 56 

skew-symmetric, 53, 325 

symmetric, 53 

transpose, 53 

triangular, 53 

upper Hessenberg, 218 

of vectors, 46 

zero, 52 
Matrix minimum method, 558 
Maximal flow problem: 

algorithm, 611, 612, 678 

basic solutions, 613 

connected subgraphs, 456 

cuts, 609 

dual problem, 610 

formulation, 607 

max flow—min cut theorem, 612, 

678 

multicommodity, 639, 676 

scaling algorithm, 617 
Maximal flow realizable network, 677 
Maximum spanning tree, 656 
Menu planning problem, 30 
Minimal-—cost flow problem: 

algorithm, 474, 478 

basic characterization, 461, 504 

formulation, 453 
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initial solution 475 
lower—upper bounds on arc flows, 
478 
simplex tableau, 481 
Minimal-—cost—maximal flow problem, 
672 
Minimal capacity, 661, 676 
Minimal cut, 609 
Minimal forward cut, 676 
Minimum ratio test, 121, 280 
Minimum weighted matching problem, 
536 
Modeling, 7 
Modeling languages, 257 
Multicommodity: 
basis characterization, 649 
decomposition algorithm, 641 
maximal flow problem, 640, 680 
minimal—cost flow problem, 639, 
640, 642 
minimal disconnecting set, 676 
transportation problem, 381 
Multi—criteria shortest path problem, 
679 
Multiobjective program, 195 
Multiterminal network, 657 


Negative cost circuit, 632, 634 
Nested decomposition method, 384, 
385 
NETGEN, 511, 605 
NETOPT, 511, 605 
Network analysis, 654 
Network design, 654 
Network, see Graph 
circulatory, 568 
connected, 456 
directed, 453 
generalized 494, 512 
hidden, 512 
Network flow problem, 453, 567, 608, 
620 
Network flow with gains problem, 494, 
512, 558 
Network simplex algorithm: 
computing: 
basic solutions, 466 
dual variables, 469 
determination: 
entering variable, 469 
exit variable, 472 
initial basic feasible solution, 475 
labeling algorithm for, 482 
list structures for, 482 
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lower—upper bounds, 478 
pivoting, 472 
tableau associated with, 481 
Network synthesis problem, 607, 654, 
680 
New activity, 301 
NEXT list, 635 
Next node, 483 
Nodal balance, 454 
Node: 
adjacent, 455 
capacitated, 501 
definition, 453 
end, 457 
from-, 455 
intermediate, 454 
leading, 474 
leaf, 457 
potential, 470 
rank (degree), 457 
root, 461 
to—, 455 
transshipment, 454 
Node-arce formulation: 
maximal flow problem, 608 
multicommodity minimal-cost flow 
problem, 641 
Node-arc incidence matrix, 459 
Nonadjacent extreme point methods, 
149 
Nonbasic 
matrix, 61, 95 
variables, 95 
variable—space, 104 
Nonbinding constraint, 71 
Nonbreakthrough, 577, 586, 594 
Nonconvex, 64 
Nondegeneracy, 95, 222, 557 
Nondegenerate basic feasible solution, 
95, 222 
Nondegenerate iteration/pivot, 107 
Nonnegativity constraints, 2 
Nonsimple path, 456 
Nonsingular matrix, 56 
Normal to hyperplane, 48, 65 
Norm of vector, 47 
Northwest corner rule, 522 
Notation, 27 
NOW list, 635 
NP-complete, 616 
Null space, 407 
Number of basic feasible solutions, 97, 
224 
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Objective contour, 19 
Objective function: 
definition, | 
parametric, 312 
Phase I, 154 
Phase II, 155 
piecewise—linear and concave, 315 
piecewise—linear and convex, 318, 
503 
unbounded optimal value, 21, 27, 
117, 265, 267, 280, 288 
value, 2 
One-arc, 545 
Open halfspace, 237 
Open interval, 29 
Open set, 87 
Optimal (basic feasible) solution, 21, 
104, 225 
Optimal control problem, 9 
Optimal extreme point, 91, 114 
Optimality conditions/criterion, 18, 21, 
25, 114, 237, 242, 280, 288 
Optimal location problem, 29 
Optimal rounding procedures, 409, 
410, 439 
Optimal solution set, 20 
Optimality gap, 618 
Optimization vs. decision problems, 
396, 436 
Oracle, 295 
Origin, 46 
Origin—destination matrix, 675 
Origin in transportation problems, 513 
Orthogonal, 48 
Out—degree of node, 457 
Outer linearization, 374 
Out-of-kilter algorithm: 
algorithm, 573, 586, 605 
arc, 570 
dual of, 569 
dual variable change, 577 
finite convergence, 585, 591 
flow change, 574 
formulation, 567 
kilter number and states, 571, 572 


Packed form, 206, 257 
Parallel computations, 256 
Parametric analysis 
of cost vector, 312 
of right—-hand-side vector, 313 
shadow prices, 318 
Pareto—optimal solution, 8, 679 
Partial pricing, 206, 220, 256 
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Partition: 

basic feasible, 222 

strongly feasible basis, 182, 229, 

488, 491, 564 

Partition matrix, 53 
Partitioned shortest path algorithm, 635 
Partitioning method, 371 
Path—-following algorithms, 451 
Path in graph: 

closed, 456 

definition, 455 

nonsimple, 456 

simple, 456 
Payoff matrix, 324 
Penalty function, 591 
Perceptron—like algorithm, 452 
Perfect competition equilibrium, 38, 

324 
Performance guarantee, 393 
Permutation matrix, 214 
Permutation structure, 178, 198 
Perpendicular, 48 
Persistency, 7 
Personnel training problem, 30 
Perturbation: 

of cost vector, 312 

of right-hand-side vector, 315 
Perturbation method, 196, 328, 557 
Perturbed KKT conditions, 432 
Phase I method, 198 
Phase I problem, 155 
Phase II problem, 155 
Piecewise-linear objective function, 

315, 318, 503 
Pivot: 

block, 134, 599, 613 

column, 250 

definition, 106, 127 

element, 127 

matrix, 214 
Player: 

column, 324 

row, 324 
Pointing toward root, 488 
Polyhedral: 

cone, 70, 71 

set, 70 
Polyhedron, 70 
Polynomial complexity: 

definition, 394 

genuine (strong), 396, 450, 617 

issues, 39 
Polynomially bounded, 394 
Polynomial-time: 
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algorithm, 81, 394, 396, 546 
primal simplex, 512 
rounding scheme, 409, 410, 442, 
451 
scaling algorithm, 617 
Polytope, 70, 75 
Positive definite matrix, 438 
Post-optimality analysis, 296 
Postorder traversal, 483 
Potential function, 418, 420 
Predecessor index, 482 
Predictor—corrector algorithm, 435 
Preemptive priority: 
approach, 195 
equivalent weights, 195 
Preflow—push strategy, 617, 619, 678 
Preorder distance, 483 
Preorder traversal, 483 
Price (fair), 272 
Price—directive decomposition, 339, 
373, 392 
Price—quantity equilibrium, 39 
Pricing, 121, 206 
Primal: 
breakthrough, 594 
feasibility, 239 
problem, 2, 259 
simplex method, 108, 121, 220 
Primal—dual method: 
case of unbounded dual, 288 
development, 286 
dual of restricted primal problem, 
287 
Hungarian algorithm, 535, 541, 564 
modifying dual solution, 287 
out—of-kilter method, 567 
path—following methods, 450 
restricted primal problem, 286 
summary, 288 
tableau format, 290 
Primal—dual relationships, 264 
Product form of inverse, 140, 207 
Production—inventory problem, 37 
Production scheduling problem, 9, 31, 
33 
Production-transportation—inventory 
problem, 505 
Programming problem: 
generalized linear, 385 
large scale, 339 
linear, | 
Projective: 
algorithm, 450 
transformation, 404 
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Project management, 669 

Project selection problem, 34 
Proportionality assumption, 3 
Pseudoflow, 593, 619, 679 
Pseudo—polynomial, 396, 617 
Pseudorooted spanning forest, 496 
Purification scheme, 408, 410, 451 


QR factorization, 443 


Rank: 
of matrix, 61 
of network flow matrix, 459 
of node, 457 
of transportation matrix, 517 
Ranking extreme points, 196, 199 
Ray, 66, 118 
Recession: 
cone, 74 
direction, 66, 74 
Rectilinear distance, 30 
Reduced assignment matrix, 538 
Reduced cost coefficients, 104 
Reduced matrix, 538 
Redundant: 
algebraically, 162 
constraints, 61, 71, 73 
geometrically, 71, 190 
Regression equation, 207, 256 
Regularizing, 414 
Relationships: 
primal vs. dual objective values, 
264 
primal vs. dual problems, 60, 265, 
266 
RELAX, 511 
RELAX-IV, 605 
Relaxation: 
algorithms, 593, 605 
Lagrangian, 371, 373 
strategy, 372, 418 
Relaxed master program, 372 
RELAXT, 511 
Replacing vector in the basis, 50 
Representation of nonbasic vector, 132, 
465, 520 
Representation of polyhedral sets, 75 
Representation theorem: 
bounded sets, 75 
unbounded sets, 76 
Requirement space, 22, 23, 25 
Rerooting, 487 
Residual capacities, 595, 618 
Resolution theorem, 77 


Index 


Resource-directive decomposition, 
373, 392 
Restricted primal problem, 286 
Restriction strategy, 405, 418 
Restrictions, see Constraint 
Return arc, 608 
Reverse arcs of a cut, 609 
Reverse cut, 609 
Reverse shortest path problems, 679 
Reverse-star, 455 
Reverse thread index, 483 
Revised simplex method: 
comparison with simplex method, 
205 
summary, 201 
tableau, 202 
Right-hand derivative, 604 
Right-hand shadow price, 275 
Right—hand-side column, 126 
Right—hand-side vector, 2, 126 
RNET, 511, 605, 678 
Robust shortest path problems, 679 
Rocket launching problem, 35 
Root arc (root), 461 
Root node, 456, 461 
Rooted spanning forest, 463, 504 
Rooted spanning subgraph, 462 
Rooted spanning tree, 461, 518, 650 
Rounding scheme, 409, 410, 442, 451 
Row generation technique, 372 
Row operations, 54 
Row rank, 61 
Row vector, 45 
Row zero, 126 


Saturated arc, 594, 665 
Scan eligible list, 628 
Scalar multiplication: 
of matrix, 52 
of vector, 46 
Scaling, 219 
Scaling method, 617, 618 
Schwartz inequality, 47 
Self—dual algorithm, 333 
Sensitivity analysis: 
addition: 
new activity, 301 
new constraint, 302 
bounded variables case, 332 
change in: 
constraint matrix, 298 
cost vector, 296 
right-hand-—side vector, 297 
tolerance approach, 308 


Index 


‘bounded, 20, 75 
convex, 64 
operations, 29 


Shadow prices, 271, 273 


under degeneracy, 273 
right~hand/left—hand, 275 
via parametric analysis, 318 


Sharp labels, 630 
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pivoting, 127 

polynomial-time, 512 

primal, 108, 121 

primal—dual, 288 

revised, 202 

tableau format, 125, 129 

transportation, 522 
Simplex multipliers, 121, 267 
Simplex path, 107 


Shortest path: Simplex set: 
algorithm: definition, 107 
arbitrary costs, 625, 630, 633, volume, 437 
635 Simplex tableau: 


nonnegative costs, 620 
approach—dependent, 639, 679 
arborescence, 622 
bicriteria, 679 
complexity analysis, 622, 630, 633 
dynamic problems, 679 
dynamic stochastic problems, 639, 

679 
extensions, 639 
flow augmentations, 679 
interpretation of primal simplex, 

622, 632 
label—-constrained, 639, 679 
multi-criteria, 679 
partitioned method, 635 
polynomial-time algorithms, 635 
problem, 547, 600, 617, 679 
reverse path problem, 679 
robust problem, 679 
stochastic, 639, 679 
time—dependent, 639, 679 
time-restricted reverse problem, 

680 
threshold partitioned method, 637, 


bounded, 225 
dual, 279 
interpretation of entries, 131 
network, 481 
primal, 125 
transportation, 535 
Simultaneous equations, see System of 
linear equations 
Single artificial constraint technique, 
293 
Single artificial variable technique, 
173, 198 
Single commodity flow problems, 639 
Single node ascent step, 605 
Single node breakthrough, 594 
Singular matrix, 56 
Sink, 454 
Size of an instance, 394 
Skew-—symmetric matrix, 53, 335 
Slack variable, 4 
Sliding objective method, 424 
Solution: 
basic feasible, 62, 95 
efficient, 8 


679 feasible, 2 
tree, 622, 637, 679 optimal, 18 
Signature algorithm, 565 Pareto-optimal, 8 
Simple path, chain, circuit, cycle, 456 space, 18 
Simplex method: Solvers, commercial, 257 
bounded, 220, 230 Source, 454 
column method, 250 Space: 
complexity, 397 Euclidean, 48 
dual, 279 requirement, 22, 23, 25 
finite convergence, 122, 181, 285 solution, 18 
implementation remarks, 180, 219 Spanning: 
initial solution, 129, 151, 293, 475, forest, 457 
489, 522 set, 49 
interpretation through KKT subgraph, 456 
conditions, 242 tree, 456, 518 
network, 466 Sparse, 206, 219 


optimality criterion, 104, 225, 280 Special structures, 339 
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Spikes, 219 
Stabilized column generation, 344, 
374, 391, 392 
Stage, 181, 491 
Staircase structure, 384 
Stalling: 
definition, 175, 180, 181 
in networks, 488, 493 
prevention rules, 491 
stages, 181, 491 
Standard form: 
of dual, 260 
of linear program, 5, 6 
State variable, 9 
Steepest edge selection, 219, 256 
Stepping stone method, 564 
Street distance, 30 
Strict complementary slackness 
property, 336 
Strict convex combination, 64 
Stroke encoding, 396, 617 
Strong duality, 266 
Strongly feasible basic partition, 182, 
229, 488, 489 
Strongly feasible trees, 489, 491, 564 
Strongly (genuinely) polynomial, 396, 
450, 617 
Strong theorem of complementary 
slackness, 336 
Structural: 
constraints, 2 
variables, 2 
Subgraph: 
definition, 456 
induced by node set, 456 
maximal connected, 456 
proper, 456 
spanning, 456 
Submatrix, 53 
Suboptimization, 220, 256 
Subproblem, 339, 342, 356, 363, 372 
Subtree rooted at a node, 482 
Successive linear approximation, 4 
Successive shortest path algorithm, 
546, 562 
Successor nodes, 482 
Sum vector, 46 
Supervisor's principle, 268 
Supporting hyperplane, 88 
Surplus variable, 4 
Symmetric matrix, 53 
Synthesis problem, 607, 654, 680 
System of linear equations: 
basic solution, 62 


Index 


dependent, 61 

Gaussian reduction, 56, 63 

Gauss—Jordan reduction, 56, 62, 
212 

general solution, 62 

number of solutions, 62 

redundant, 61 

solving, 55 


Tableau: 
assignment, 538 
bounded simplex, 225 
dual simplex, 279 
primal—dual, 290 
revised simplex, 202 
simplex, 125, 481, 535 
transportation, 514 
two-phase, 164 
Tangential approximation method, 374 
Tangential support, 374 
Tanker scheduling problem, 13 
Technological coefficients and 
constraints, 2 
Termination criterion: 
infeasibility, 21, 22, 24, 155, 169, 
170, 280, 288 
optimality, 18, 19, 114, 169, 242, 
280, 288 
unboundedness, 21, 27, 92, 117, 
169, 231, 280, 288 
Theorems of the alternative, 235 
Thread index, 482 
Threshold partitioned shortest path 
algorithm, 637, 679 
Tie breaking rule, 178 
Tight constraint, binding, active, 71 
Time-—space network representation, 
638 
Tolerance sensitivity approach, 308 
To-—node, 455 
Totally unimodular matrix, 463, 464, 
517 
Tourism problem, 332 
Traffic assignment problem, 37, 675 
Transposition of matrix, 53 
Transportation problem: 
algorithm, 522 
balanced, 514 
characterization of basis for, 518 
comers of cycle in tableau, 517, 
520 
definition, 11, 513 
degeneracy in, 528, 534 
properties of constraint matrix, 516 


Index 


rank of matrix, 517 
representation of nonbasic vector, 
520 
simplex tableau associated with, 
535 
starting solution, 522, 558 
tableau, 514 
Transshipment: 
node, 454 
problem, 551, 562 
Traveling salesman problem, 501 
Tree: 
correspondence to basis in network 
flows, 461, 463, 518 
definition, 456 
dominant requirement, 655, 656 
end node, 457, 518 
equivalent characterizations, 458 
maximum spanning tree, 656 
one-, 458 
properties, 457, 458 
rooted, 461, 518, 650 
spanning, 456, 518, 650 
strongly feasible, 489, 491, 564 
uniform requirement, 661 
Triangularity of basis in network flows, 
463, 518 
Triangularization, 212 
Triangular matrix, 53 
Two-person zero-sum game, 324 
Two-phase method: 
analysis, 157 
comparison with big—M method, 
168 
description, 154 
Two-stage stochastic program with 
recourse, 39 


Unary encoding, 396, 617 
Unbounded: 
optimal value, 21, 27, 92, 117, 169, 
231, 265, 267, 280, 288 
polyhedral set, 21, 76 
subproblem region in 
decomposition algorithms, 354 
Uncapacitated problem, 454 
Undirected graph, 455, 654 
Uniform requirement tree, 661 
Unimodularity, 517 
Unique: 
optimal solution, 20, 114 
solution, 62 
nit vector, 46 
nrestricted variables, 4, 147 
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Updated: 
basis inverse, 208, 307 
column vector, 209, 342 
LU factors, 217 
network list structures, 485 
tableau, 128, 227, 228, 307 
Upper bounds on variables, 220, 478 
Upper Hessenberg matrix, 218 
Upper tree, 488 
Upper triangular, 53 
Valid cut, 304 
Valid inequality, 304 
Variable: 
artificial, 153, 198, 522 
basic, 95 
blocking, 106, 112 
bounded, 220, 478 
definition, 2 
dual, 239, 260, 269 
entering basis, 106, 121, 178, 225, 
493 
flow, 454 
integer, 304 
leaving basis, 112, 121, 178, 479, 
527 
legitimate, 153 
nonbasic, 95 
slack, 4 
surplus, 4 
unrestricted, 4, 147 
Variable splitting technique, 511 
Variable upper bounding, 257 
Vector: 
basic, 49 
definition, 45 
dependent, 49 
direction, 66, 71 
eta, 208 
Euclidean space, 48 
independent, 48 
lexicographically nonnegative, 182 
lexicographically positive, 182 
norm, 47 
normal, 48, 67 
operations, 46 
sum, 46 
unit, 46 
zero, 46 
Vertex, 72 
Vertex of ray, 66, 118 
Vogel's approximation method, 524, 
558 
Volume of simplex, 437 
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Warehouse location problem, 388 

Weak duality, 264 

Weak theorem of complementary 
slackness, 268 

Weighted average, 64 

What-if analysis, 295 

Working basis, 221 


Zero: 
matrix, 52 
vector, 46 
Zero—arcs, 545 
Zero—sum game, 324 
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